[Lilug] Poor mans ETL processing

[Justin Dearing]

Peter,
Whose stopping you from mentioning it? Employer or vendor?

Also what are you ETLing from? CSV, EDI, XML, etc.

At a basic level though, you can write shell scripts pipe all those commands
to get your data in a format that can be bulkloaded into your database..



On Tue, Feb 24, 2009 at 3:21 PM, Peter Capazzi <Peter.Capazzi at tranzactis.com
> wrote:

>  At our work place we use a software that has a pretty heavy price tag.
> I'm actually not supposed to mention it by name. Anyway, the more I work
> with and the more I learn about linux I see some similarities between
> commands in linux and the functions in this application.
>
> I was wondering if there was a "poor man"s ETL engine that is simply a
> front end to commands like these:
> sort - sorting datasets on a field
> join - merging data sets
> cut - field selections/exclusions
> cat - for input files
> sed - To modify fields
> grep - To filter
> awk - To modify fields and partition datasets.
>
>
>
> _______________________________________________
> Lilug mailing list
> Lilug at lilug.org
> http://lists.lilug.org/listinfo.cgi/lilug-lilug.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lilug.org/pipermail/lilug-lilug.org/attachments/20090224/fe199b44/attachment-0003.htm>