What the Future Holds for the REAL WebMonkeys

Actually, a better title would be “What does the future hold for the real Webmonkeys?”

Tim Bray wrote up a piece about how his brother – some sort of tech manager – was stunned by the quality and quantity of resumes he got for a job posting for a relatively mediocre job: Just shows how bad the market currently is.

Don Box – a Microserf – apparently picked up on this thread and added his thoughts about this situation, which both Tim and I find interesting:

At least one blog has discussed whether IT was a worthy profession for young people to pursue.

…In general, I would be very hesitant to encourage anyone to pursue this career path. Like writing a book, the only people who should do it are those who can’t not do it. I’m hopeful however that there will be many generations worth of funding for those few people who absolutely must live in the world of executable abstractions.

That’s an interesting and provocative take on it all, and I think it sheds a little light on what’s going on now, namely the following:

  • The whole dot-boom was ignited by the propeller-heads that were doing this out of love of this sport. They had to do it. While the fuel that kept the boom rolling was $$ (VCs, dreaming of wealth, not bitwise operators), this also helped make it go “boom!” (note: it would have anyway; nothing is as good as the Net was hyped to be).
  • A lot of people laid off from tech jobs are the very people who made it all happen: techies, desk support etc. The managers go last, and they have no clue outside a Visio diagram about how the company actually works. And a lot of the other people left – tech or not – came into this game for the $$ of it, not the fun of it. So even the tech people that are left are the passionless ones.
  • A lot of people have gone into CS – as they went into Law or Medicine before – because it was a solid career move; you’d get out and be able to get a nice, well-paying job unless you were a real tool. How that has changed!
  • And let’s be honest – unless you have a manager that is tech savvy and understands what makes a tech team work well (hint: not $$; yes, it helps, but…), who is a manager going to hire: The sharp-dressed, business savvy college graduate who sprinkles his sentences with XML, outside the box, pro-forma terms — or — the passionate nerd who grew up on MUD, builds his own whitebox computers and gets excited about cutting a tenth of a second off page load time? Passion loses; we all lose.

Database Notes

Just for fun – because I’m not – I recently rebuilt an ancient box I had Linux running on.

I wiped the hard drive and started from scratch.

I’ve installed Linux a fair number of time now (always from CD; still not a full-blown propeller-head), and so it was pretty much of a no-brainer.

The interesting part – to me – was getting the two databases I installed up and running: mySQL and Postgres.

Again, I’ve installed each of these at least three times in the past (mySQL on Windows, as well). But I never brought them up at the same time, attempted to get a “production type” system in place with each of them at the same time.

So the contrast was interesting.

Some observations:

  • Ease of installation: This one is not even close: mySQL wins hands down. I’ve never had a painless Postgres installation – why do I have to build all these directories etc to start the database? Sure, if I want the database files somewhere else, I should be able to in a config file…why doesn’t this exist? mySQL, while less flexible, it more of a Windows-like install (you just install and it’s there, like where it is or not)
  • Migrating data from existing databases: mySQL wins again. While there is the file import/export for Postgres – as well as pgdump – the tools out there for mySQL (I’m running SQLyog on Win2000) make migration a breeze. The tools that are out there for Postgres either 1) Suck, or 2) Aren’t up to the level of the mySQL tools (like SQLyog). That’s a problem.
  • Security (locking down for Internet exposure): Postgres wins. The security model for mySQL – if you can even call it that is weak (GRANT….). I’m probably missing something, because I’m more of a SQL Server user than OSS database user overall, but … wow… weak. Postgres’ security is pretty damn good, which is to be expected from a database that is touted as an almost-Oracle RDBMS. Note: One of the reasons that Postgres is more cumbersome to set up is because of the security: You have to have a non-ROOT user running the server, recommended to add another user, set permissions on the database directories to reflect this etc. More work. More security. Depends what you need the database for.
  • Ease of use (once installed and configured; data set): Let’s see. Postgres is ANSI 92 compliant, an ACID database, supports stored procs etc……mySQL lacks subselects (that’s my biggest beef with mySQL) and has weird – proprietary SQL (can you say CONCAT? sure, I knew you could…). Yes, Postgres wins hands down.

But – as mentioned above – it’s what you need the database for. Like Cold Fusion, mySQL is “not up to playing with the Big Boys…”, but how often do you play with the Big Boys?

Yeah, mostly it’s whacking together a quick solution to a problem. mySQL is much faster to get up and going for simple tasks.

It’s all about the best tool for a given job. My prejudices and preferences fall in the Postgres camp, but there are times when I need something simple and need it fast.

mySQL is there for me.

And yes, it’s all about choice, as well. And with these two out there – goosing each other to some degree – we all win.

Underappreciated Web Tools

OK, we all spend a lot of (..ahem…too much) time in front of a screen or in back of a rack of servers on a daily basis, and we begin to take a lot of things computer-wise (especially code wise) for granted. We just expect stuff to work, and when it doesn’t…well, of course it’s not user error.

I was just fiddling with this and that today, and it occured to me that I take this and that for granted.

What follows is just a short, incomplete, unordered list of things that just work – yet we never stop and thank them….

Things We Ignore That Rock

  • Perl – Whenever I have a problem to quickly solve, I turn to Perl. I’m not a strong Perl coder – hey, my code is still readable, good Perl looks like “/%@=>\/\/?^” – but it’s the most useful language out there. I hate it when I’m on jobs that need a quick (or permanent) Perl solution but the company is disinclined to use Perl because of platform or politics.
  • Ethernet – The little cable that could. Yeah, wireless is cool, wireless is more flexible….Ethernet is just blazingly fast, secure, cheap (I have a boxful of old NICs, esp. on Linux, all work) and pervasive.
  • HTML – For all the bitching we all do about HTML and – especially – its limitations, well, look at all that has been done in spite of its quirks and limits. Could you have imagined this ten years ago? Browser based banking/brokerages/shopping? Come on….the bastard child of SGML has done good.
  • Windows – Let the flames begin! Like it or not, most of the world that can use a computer know only one OS: Windows (pick a flavor). Much like HTML, Windows has helped enforce (or do you say “force”?) a standard. Standards are good. And like it or not, if you need a software package, odds are your chances of finding it on Windoze is better than any other platform. Monopoly? Probably. Reality? Positively.
  • Broadband – I’ve written on this before. While most ads and people who gush over broadband gush over the multi-media capabilites and all that crap, I still maintain that the best things about it are twofold: 1) Always on; 2) Fast for ordinary tasks. (To be fair, ordinary tasks of today are the extraordinary tasks of yesterday, and I expect this trend to continue). Think about your connectivity at work – oh, wait: You don’t think about that. It’s just always there. Always on. Always fast. You kvetch if a page loads slowly… Get my point?
  • The Community – Whatever the hell THAT means. But people are so willing to put stuff out there and, while it does not supplant books or courses or mentoring, it is a great resource. Contributing to the Linux kernel, a page on how to JavaScript mouseovers, why this monitor sucks and this one doesn’t. It’s all out there for the taking. For free. Sometimes that’s all it’s worth, but…it’s there for the taking.

Zen and the Art of Web Authoring

I’m a pretty patient guy, but having worked extensively on the three major platforms (Windoze, Apple, *nix) with a number of authoring languages (ASP, JSP, PHP…) and types of Web authoring tools (Perl, XML/RSS…), I can safely say that it’s a wonder I have any hair left.

Perhaps it’s good in the “job security” way that things are still not as transparent as they should be, but it’s tough to launch anything out there.

As my last entry noted, I finally whacked together an RSS feed. (When am I going to switch to either MoveableType or roll my own blogging tool [I have on in PHP running locally]?!?)

Fine. It works. Haven’t set the cron for it yet, but whatever. (Update: Set the cron.)

But it just is so frustrating to roll RSS out.

Nomenclature: Some call their files “[whatever].xml”, some “[whatever].rss”, some “[whatever].rdf”.

How can aggregators easily find these puppies? Must make them tear their hair out. And is there going to be a standard in the near future? I.e. all RSS feeds must be “[whatever].rss”?? Let’s decide now so there is no need for backward compatability in the tools that are currently coming out, and so authors (me) can avoid renaming/recoding. Makes sense to me

And the whole RSS vs. RDF battle, the namespace nuttiness…

And on one of my (hosted) domains, I had to create a MIME type in the .htaccess file (show of hands – how many NT types even know what the heck a .htaccess file is? Hell, in Linux it’s now in magic.conf or httpd.conf – there is no .htaccess per se). Could I easily find the right type to set?

NOoooooooooooooooo….

I took the one from my Linux box; didn’t work. Googled a bit, found the application-type MIME setting; still didn’t work.

Oh, I have to rip out the encoding type from the XML header. (That was just a guess that worked) Then it worked (in IE; barfed in Netscape..)

Bleh.

Bottom line: The file was built fine; it validates and all that, but it behaves differently in different client-server combos.

This is never a good thing.

But the RSS feed is for aggregators, not for eyeballing in a browser window, so I should stop my whining, right?

Actually not: What about doing a site JUST with XML, with included XLS file? Will there be browser issues there? If so, this is a deal-breaker…

Ah well, for all the XML buzz that’s been generated over the last, what, four years, XML is still very much in its infancy. And in many ways, RSS is the first widely deployed XML tool. You’re seeing that little orange XML button all over the place (shouldn’t it really be RSS? Marked up in XML, but to the RSS spec[s]…).

As long as things are moving forward…

RSS Feed

Well, after a good deal of Perl work, I think I have an RSS feed set up for this site.

The limitations are many – only applies to the current Index page (last seven days); Blogger does not have the concept of permalinks and so on – but, hey, it works.

Now I get to see if it works with this, the test entry. If it works, I set a CRON and see it I can keep it fairly current.

Note: I’ve already added it to an aggregator, and the damn thing works.

Yes, limitations. But hacking around them…

Interesting Time on the Net

Many years from now (yes, in a galaxy far, far away or whatever…), I think a lot of us will look back on the last few years as a Golden Time on the Net.

In some ways, the “golden” is actually pyrite, but that’s often the case, isn’t it?

We are reaching the fulcrum of the Web’s use; shortly, what is now exciting/evil will either become normal or cease to exist – or, more likely, morph into something that makes it almost difficult to recall the excitement/anger the original stuff created/caused.

In many ways, we have crossed over the fulcrum – the Web, in it’s current incarnation, is a very user-friendly place. Like it or not, IE is the overwhelmingly dominant browser, so designers are designing for it in a consistent manner. (No, this is not a good thing, but it is consistent, which is good.).

The geekiness needed to navigate the Web is fading away. For the average user, even a dial-up connection is not too intimidating, and getting from here to there is becoming a no-brainer.

Other areas, however, are still firmly entrenched in geekDom, but this is starting to go away, as well. Some thoughts on what one finds/what defines the fulcrum of Net use:

  • Ease of use – A better title would be Transparency, where the user doesn’t have think about how to accomplish a task. Yes, operating a normal phone initially required a learning curve/process, but who thinks about it now? And of course people have answering machines…. (Don’t get me started on the feature-bloated, user-unfriendly cell phones…). As mentioned above, we are pretty much there with the Web itself (past the halfway mark, let’s say); other areas leave a lot to be desired. A very good (and damn early) example of Transparency is DNS – geeks grok it; most don’t even think about it (and it they did, the wouldn’t 1) Care, 2) Get it). Yeah, I can remember www.yahoo.com…could I remember 66.218.70.49? And let’s not even discuss dynamic IP addressing..
  • Commonality – The Transparency driver, this is losely defined as a baseline set of common elements/behavior between uncommon systems. So one can move from one to another without really having much of a learning curve. (See Accepted Standards, below) It’s like going – today – from a Mac to PC or vice-versa. Have to learn new habits for keystrokes, there is a finder/no Explorer and so on, but the basic concept is the same. With KDE especially, Linux on the Desktop is getting close to being possible (main roadblock: Lack of apps for Linux. Photoshop; PeachTree Accounting; PowerPoint; Visio…). This applies to computers only right now – probably leaving out PDAs, even – but should embrace all tools in the future, including such non-PC devices such as TiVo and XBoxes.
  • Accepted Standards – I don’t mean this in the way there is a push for Web Standards (which I’m all for, by the way). By accepted standards, I mean basic constructs/uses that are expected. It’s in many ways an extension of Commonality, which…enhances Transparency…hey, this is actually making sense! Example: In this past week, the Net has been festooned with all sorts of e-mail viruses and worms. A lot of the discussion of this surrounds things like “well, use SpamAssain” or “IMAP beats POP/SMTP” etc. Outside of the geeks, to whom does this make sense (hint: no one). In the future, all e-mail clients will come with built-in Baysian filtering that is not identified as such (because it means nothing to most) and defaults pretty much the same for all e-mail tools and has similar (Commonality) preferences options so those who want can tweak. For most users, the default is the 80/20 solution. Note: Accepted Standards will shift with the times and technology. Commonality today might mean a standard type of dialer; tomorrow it will mean every electronic device is Web enabled (wireless or hard-wired; same behavior). The day after tomorrow it mean it will all be in 3D and make lattes for you… Also note: The defaults I allude to will be the security holes of the future, much like Windows + Outlook is a security target today simply because it’s the biggest bang for hackers buck.
  • The Ho-Hum Effect – As things become more standardized and transparent, the “wow” effect of the early Web will, for the most part, disappear. This is at the same time sad and a good thing. While there will be new ideas that suddenly enliven the Web (for short times, in some cases), most of the buzz will still be from the Geeks. Ordinary users will not see them. Think about cars – no, you really don’t think about cars, do you? But, hell, there are all sorts of new, space-age, Web-based improvements going into cars that I can see one getting excited about them. But I’m not. My biggest car thoughts are when to fill my tank. That’s it. We are beginning to see the ho-hum Effect in progress on the Web with Blogs: Once-crazed bloggers are getting tired of doing so, or finding the reasons to blog less interesting. Examples:

    • Megnut (yes, a founder of Blogger, no less!)
    • Jeremy Zawodny – A techie at Yahoo!; great blogger…
    • Kottke.org – One of the first bloggers I read consistently; this is his most recent blog…four days ago…hmmm…used to be four posts a day…..

    Sure, these are just anecdotal, but these are hard-cored bloggers and now…the thrill is gone to a degree. Which is fine. It’s part the the move to the fulcrum (and the sheer number of blase blogs…)

So this is a fun time for the Web: Maddening (Windows updates/patches, worms), exciting (what will blogs become, if anything?), uncertain (I keep hearing about Web Services. Don’t see them. What should I learn? Java or .Net?).

And – at the same time – the Net is still much in its infancy. Well, perhaps it’s getting out of the terrible teens is more accurate, and trying to decide what it wants to do when it grows up (IPv6? Is Google God or Evil? How to solve the last mile problem?).

And for any of us with any interest or ties to Open Source Software, there is the delicious, Alice Down the Rabbit Hole drama that is SCO vs., well, everyone from the sound of it. (Read – daily – the coverage on GROKLAW – you can’t make this shit up…)

But I could be wrong. Maybe the Net is just warming up…

I could live with that.

Spam…spam…spam…spam…

Looking at my inbox this morning – about all spam, half of which was herbal/generic Viagra ads, the other half diet pills – I recalled the Jefferson Airplane song of oh-so-long ago:

One pill makes you larger,

And one pill makes you small…

— “White Rabbit”, Surrealistic Pillow album (it was an album, not a “CD” back in 1967

Outage

My broadband connection was out all day yesterday (I was at job during the day, but coming home to … no Net).

Painful.

Didn’t realize how much I relied on it/was addicted to it (depending on your perspective/politics).

Back now.

Good.

Blogroll me baby…

I’ve finally gotten around to adding a blogroll to my pages; I’ve been procrastinating on it simply because I’ve been trying to find a hack that would enable me to not hard-code the list.

I have not been able to find that hack, dammit!

So my (incomplete) Blogroll is now out there.

Still, it doesn’t seem right to have it static….

More on Horsepower

I was just reading an (old) blog entry by Jeremy Zawodny about how Yahoo! (where he works) got Slashdotted.

OK, as the commenters pointed out, it was not Slashdotted – taken down by the deluge of requests brought on by a Slashdot post.

But that was Zaw’s point: He always reads about how sites get taken down by the Slashdot effect, and even though Yahoo! got enormous traffic due to the /. posted, it was no biggee.

For one box, running FreeBSD and Apache. (ONE box folks).

His points on why other sites go down with high traffic were just two:

  1. Lack of sufficient bandwith (Obviously, not a problem for Yahoo)
  2. Dynamic content

I totally agree with him, but the second point made me think, especially in light of my previous post.

For those of us with no lives (yes, we know who we are…), it’s common knowledge that one of the secrets of Yahoo’s stability and speed is it’s use of static content – the top page and other indices are rebuilt on remote boxes every [what time frame?] and pushed to the front-end Web servers.

So it’s all static content. Very fast.

Which totally made sense in the early days of the Web, with the 9600 baud (and less) modems, slower home and server boxes (WOW – a 166Mhz Pentium with 96M RAM! That must smoke!!!). Take away as much overhead – an/many OBDC request(s) – as possible and the site will be faster.

Yet this was, at the same time, the era of the static Web. While Yahoo! was clever in making as much of what would normally be a dynamic site static to improve overall performance, Yahoo! was the type of site – a search engine (more accurately for Yahoo!, an indexing site) – that was one of few that needed to be dynamic.

Back then, the only dynamic nature of sites was either via SSI or Perl CGI scripts. And they were used, generally, for things a static site could not do: Page counter, quizzes, creating pages for and storing info from form pages. (The one exception was the use of SSI for header and footer files, so site maintenance could be somewhat more manageable.)

Most sites were collections of static, hand-coded pages.

Need to change that header graphic? Search and replace all 415 pages, dude….

Then – sometime around the time of the dot.com upload – the concept of portals emerged as a business model that would make those companies/investors filthy rich.

The key of a portal was all about personalization: Ralph goes to Excite.com and gets one look/feel/content; Rebecca gets another. Same site.

Suddenly, terms like “sessions” and “database” entered the vocabulary of a lot of non-techie folk.

At the same time, hardware was getting faster, backbones were getting more robust, and connection speeds were getting up to (after the insane standards battle- K-Flex vs. [I fergit – X?]) 56K! Some folks even had ISDN, which was twice that rate. Whoo hoo!

Suddenly, regular – non Web-only – sites (think “The City of [your hometown]” Web site) startied either embracing the personalization trend to attract more customers/eyeballs (remember “attracting eyeballs”?) so they could, too, become filthy rich, and to provide a way to better manage the site.

For larger sites, true content-management tools came out (think Vignette); for smaller sites, it was a small developer/development group building a narrowly defined content management toolset.

Either way, a fundamental shift was underway: Static pages gave way to templates, which pulled – at the time of request – the appropriate data from a database to provide that page’s content (or portions thereof).

We have been moving in that direction ever since.

While the portal concept has died down (lives on, in a better positioned way, on company intranets), the concept of a database-driven Web site is the norm now (who da heck has a truly static, hand-coded site anymore? Home sites, sure, but few other types).

In fact, it’s gotten a little bit nutty lately, where you look at code for some sites (including those I’ve built….) and you see queries tossed around like rice at a wedding. A query to set this, to set that…but that’s another rant.

Basically, the Web is currently, overwhelmingly database-driven.

Which gets us back to Zaw’s second comment: Essentially, dynamic pages incur much more overhead than static pages.

Again, agreed.

Yet, with all this bandwidth and turbocharged hardware, do we really care about this anymore? And – even if we do – computers are going to get faster and more people are going to get broadband (which will get “broader” as time goes by…) and so on.

So, should we really care about this? I mean, when is the last time you really worried about file size, image loads and so on. Sure, that’s hard-wired into us now, but the things I see being done every day are things that would have been red flags not too long ago.

Current Standard Web Practices – or, “what would have gotten your knuckles rapped not too long ago…”

  • Deep table nesting: Dreamweaver is, in my mind, responsible for this. I’ve gotten “templates” from designers with HTML that is so nested even I have trouble figuring it out. A table with a TD with a table making up the cell…and cells in that table each have tables…with rowspans/colspans…..Sometimes, I just look at the output (in a browser) and re-code the HTML. It always comes out cleaner, easier to maintain, and with much less nesting.
  • Massive image pre-loading This is usually due to loading images used for navigation (both “on” and “off” images). Yes, I know, CSS can do this; tell them (and CSS does not work (well) in Netscrape 4.x…kill me now…)
  • Lots of images: While most places run image through ImageReady (it’s like magic..) or similar tool, there is still no issue with putting X images on a page. No real examination of of the cumulative effect of all these images. Discussions often focus on whether to display (catalog example) the thumbnail or full-size image on a page; this discussion is invariably about appearance and number of catalog entries per page, not about load time.
  • Flash: While a great tool, it’s overused, much like organizations overused fonts when desktop publishing became commonplace. Didn’t need 13 fonts on a page, but now you could do that….. And while Flash is vector-based and usually pretty small; folks, it’s the startup that kills you. The load, fire up that plug-in (remember, it’s not native) that is the overhead, that can cause page-load slow-downs and rendering issues (esp. in IE6, I’ve noticed)
  • Standards:
  • Standards? We don’t need no stinkin’ standards?! While CSS has gained a foothold, and tools like Dreamweaver have (by default, so users are unaware) enforced standards (close LI and OPTION tags, for example), most developers are clueless about XML, XHTML, HTML 4.x. I guess the tools have to catch up.

  • (trust me, I could go on and on…)

Note: This is just what I see happening at a lot of development houses. While progress is, well, progressing (use of image-optimizers, CSS has almost made extinct), there is still work to be done.

So should we care about static (fast) vs. dynamic (not as fast)?

I dunno.

It’s interesting, because at the same time that our hardware/connections are becoming so fast that upgrades make virtually no difference, I’m seeing a trend toward static publishing.

Rather, a Yahoo!-style publishing, where dynamic content is written out, and then served as static pages.

Case in point: Blogs.

Blogs are a typical content-managed solution: Templates are tweaked for visual representation; content is stored in a database. All content changes are made via a back-end tool that stores info in a database.

Very familiar, so far….

But here comes the twist: In most cases (let’s say, the default setting on most blog tools), the data is then written to static pages, which is what is ultimately served up by the Web server.

So while the site is dynamic in that there is databased content, the content the end user sees is not pulled from the database at run time.

Interesting.

But is it necessary?

OK, for tools like Userland Radio and the tool I’m currently using (basic Blogger), it’s necessary because the database does not exist on the Web server. Created in one place (with resident database), and then pushed to Web server as static pages.

Works well; makes it tough to integrate “run at run-time” tools, but, yes, snappy.

But – except for the tools that make it necessary to create static pages – are static pages really necessary?

Again, I dunno.