Systems Journal / Kirk Sweeney

Data solutions for a modern world.

Notes from building data platforms and software: practical patterns, sharp edges, and things worth learning in public.

Recent writing

All posts →
6 min read

Starting with (py)Spark

There has been a large shift in the industry in recent years to moving to python for data engineering. Spark one of the leading distributed data processing engines has great suppor