Blog This!   Lee Geistlinger's Web Log

Blog Home
Blog Archives
LittleGhost Home

E-mail: geistlinger AT gmail.com

Everyone loves lists Books, Movies and so on

My Online Aggregator

OK, I was thinking about databases.

What am I thinking about now?

I've been coding my brains out lately, but in a very helter-skelter way that (occasionally) dovetails nicely.

The following is a list of what I've been working on lately:

• Littleghost.com: Revamping my littleghost.com Web site for the first time since launch. See earlier entry.


• HTML 4.01/XHTML: Now that Netscrape is finally standards complient, it's time to really knuckle down and figure out how to use the tools the W3C has given us over the years that we just could not use effectively. This is a large part of the Littleghost.com redesign. (Note: A recent survey [by who?] said that IE 5-6 have 95% of the market. Fortunately, Netscape's offerings were strongest with their new offerings, NOT v4.x).


• Perl & PHP: For various reasons for various tasks, I have been doing a lot of Perl and PHP. I like both a great deal. I have been working with both languages for about 2-3 years, but never really got a lot of time to use them. I'm making time now. I wish my providers supported PHP (one does, but have to code them with a she-bang like Perl scripts and put in the CGI-BIN, which makes them fairly non-portable)


• Web services: As mentioned in an earlier entry, this is something that I got into because Amazon and Google are opening their APIs to a degree, and use of XML tools make both sites accessible. No linking; no frame-out. Import the raw data and knock yourself out...


• Javascript/DHTML/CSS: As part of my "standards" search/pursuit, I've been doing a lot of this, and making sure it works in IE and Netscape. For the last year or so I've been designing for IE solely, and there are still some quirks required to make anything the same in Netscape, even the new versions. So -- OK -- the true "standards-complient" browsers are not here yet, but they are getting damn close. Thank god the damn LAYER tag is gone....

I've been doing a little XML, some Cold Fusion, some stored procs and messing with three different databases (mySQL and Postgres on Linux; MS SQL Server on Win2000), as well. Probably not as much as I should, but there is only so much time.

=======================

One other thing I have been getting into lately is shell scripting. I finally found a book (PDF, on the Web, free) on BASH scripting (I use Bash on my Linux box; to be honest I don't know if I have the Korn or Bourne shell on there. Doesn't look like it).

Shell scripts are a pain in the ass, but excellent coding practice. They are difficult because they are so precise. With HTML, you can get away with almost anything (no close TR? the browser understands). With Cold Fusion, you get away with a lot (not case sensative, loosely typed etc). With Perl, it'll slap you for case, but other matters are handled transparently (variable $num not exist? Then "$myNum = $num + 7" will equal 7. No error).

Shell scripts require all sorts of rule-following, the most difficult -- to me -- is the space issue:

I like writing: $c = $b + 4;



Shell scripts require no spaces: $c=$b+4;

Yeah, same thing, but .... just not my usual coding practice.

But good -- you HAVE to be precise with shell scripts, which is a good thing. (However painful)

In a case of my "learnings" dovetailing, what I'm doing with the shell scripts is writing scripts to back up important files/directories on my Linux box to the Win2000 box and vice versa. This required the following tools/skills:

• Shell scripts to do all the work, which includes FTP get/puts and so on


• Installation/administration of an FTP server on my Win2000 box (freeware)


• Installing the command line tool for WinZip, so I could write batch files to zip up selected directories


• Scheduling -- on the Win2000 box -- the Zip batch files


• Scheduling -- via CRON -- the jobs on the Linux box (all jobs run off the Linux box except the Win2000 directory zip, which are batch jobs. Linux is much better for scheduling and scripting [have tar, gzip, permission handlers etc all at your fingertips] )

It's been an eye-opener.

I currently have eight CRON jobs running every night; before the crons run, I have two scheduled batch jobs on the Win2000 box zip things up.

Pretty cool.

And the best part is that I wrote these a month or so ago, and I just let them go. And they keep working. (Yes, I do check that they ran, and occasionally try to "restore" from a backup: never failed yet).

This was a lot of work -- simply because a lot wasn't in place (FTP server etc), but because I do have at least passing familiarity with the crontab, scheduler and so on, it was pretty straight-forward. Lot of work; lot of time -- but no "deal breaker" dead ends. Just busy work, to some extent. I would figure I'd need this or that; I'd do it. No biggee.

Sometimes being an inquisitive geek pays off.


I wrote -- over a year ago -- that I was "thinking about databases" and all that.

That train of though turned into a guest editorial on the subject of open source databases vs. commercial products.

It was interesting to write -- made me think -- and, of course, the response from readers was the really interesting part.

Sure, I got flamed, called an idiot and all that, but there was a lot of knowledge and experience behind the responses in many cases.

Basically, the article said "Open source solutions are in many areas comparable or better than commercial products, but this is not true in the case of OSS databases. Why no outcry (or am I missing the outcry)?"

And -- basically -- the response from readers was that what is out there is fine; the options offered by commercial products were just not needed or could not be cost-justified.

Wow. Blew me away.

Because the most widely accepted/deployed OSS database -- mySQL -- is really a piece of crap. It doesn't pass ACID tests, it's filled with proprietary (instead of ANSI-complient) SQL (such as the "concat" operator! Scary...), and does not support a lot of the things that make it a database.

While there were dissents -- and those who said that, yeah, Oracle is good, but I'll never use the 10 million configuration options offered (fair) -- the general response was that mySQL is just fine for the job.

Basically, people are using mySQL -- and other OSS databases, such as Postgres, SAP etc. -- much like flat files. Just a big table or two; maybe joining the two in some cases. Very denormalized. The advantage to using a database instead of a flat file even in this case, of course, is that one doesn't have to write the logic to extract/order/limit the data pulled from the "data store" -- SQL is used.

And then you can extend it later -- add another table etc -- very easily. And -- importantly -- without changing/adding any business logic.

And that is a good thing.

But it was just a bit scary to me: I had thought that the OSS crowd, in general, was more sophisticated about databases than that. I got notes and there were posts from people who had been doing this for years and saying, basically, that they don't use primary keys and so on.

Again, the "relational flat file" syndrome.

While I agree that many projects do not need the weight of an Oracle or MS SQL installation, but one should still adhere to good database design and usage no matter the product used. It just seems odd -- and surprising -- to me that the users of OSS software don't seem to put a lot of stock in these "best practices."

On the other hand, I'm judging from the people who posted. And those who posted -- or wrote -- are probably always going to be those who disagree, not the ones nodding their heads and thinking "yep....".

All in all, an interesting project (the article/responses) overall.