The Zen of PyFlink: The evolution towards a truly “pythonic” Flink

Have you been a python data streaming engineer who has invested hundreds of hours tracking down errors, going back to documentation? Does your stream processing pipeline go through slow iteration and abandonment because of frustrating results not meeting expectations?

Then you are in the right place, where we talk about a new idea around the PyFlink Table API which is aligned with the APIs that Python developers love, like Pandas, Polars and PySpark. There have been some gaps in the PyFlink API and some mismatches that made applying PyFlink stream transformations a bit off. In this session, learn about the changes being proposed to improve the open source PyFlink Table API and make it the stream processing framework of choice for data streaming community!

This talk details the findings, improvements and plans being made in the PyFlink open source community with support from Confluent, Alibaba, OpenAI and others to move beyond the hard edges and idiosyncrasies of Apache Flink’s established JVM ecosystem. Join us to learn how to contribute and make PyFlink a truly loveable product.