Skip to content Skip to sidebar Skip to footer
Showing posts with the label Luigi

Luigi: How To Pass Different Arguments To Leaf Tasks?

This is my second attempt at understanding how to pass arguments to dependencies in Luigi. The firs… Read more Luigi: How To Pass Different Arguments To Leaf Tasks?

When A New File Arrives In S3, Trigger Luigi Task

I have a bucket with new objects getting added at random intervals with keys based on their time of… Read more When A New File Arrives In S3, Trigger Luigi Task

How To Continously Update Target File Using Luigi?

I have recently started playing around with Luigi, and I would like to find out how to use it to co… Read more How To Continously Update Target File Using Luigi?

Recurrent Machine Learning ETL Using Luigi

Today, running the machine learning job I've written is done by hand. I download the needed inp… Read more Recurrent Machine Learning ETL Using Luigi