[Lilug] Poor mans ETL processing

[Peter Capazzi]

Hi Matt,
Yes that's true, I was focusing on the Transforms, simply because that's
where the bulk of my work is. The Extract and the Load would work with
either cat or freetds/sqsh (to pull from SQL2000). 

Peter Capazzi wrote:
> At our work place we use a software that has a pretty heavy price tag.
> I'm actually not supposed to mention it by name. Anyway, the more I 
> work with and the more I learn about linux I see some similarities 
> between commands in linux and the functions in this application.
>  
> I was wondering if there was a "poor man"s ETL engine that is simply a

> front end to commands like these:
> sort - sorting datasets on a field
> join - merging data sets
> cut - field selections/exclusions
> cat - for input files
> sed - To modify fields
> grep - To filter
> awk - To modify fields and partition datasets.

Heh ... I think they call it "perl" :-D

Seriously, though, I guess what you are looking for above is a lot of
the "T" (transform); do you also need the "E" (extract) and "L" (load)
as well?  Although I don't have any recommendations, perhaps that
information can help guide others to suggest alternatives to what you
currently have.

Cheers
Matt S
LILUG Secretary