An often encountered Antipattern in Python code are complex queries composed with the help of string substitutions and concatenations. SQLAlchemy Core allows you to generate queries in a programmatic way for many SQL databases like BigQuery.

A new star is rising in the Python ecosystem. Polars is a blazing fast DataFrames library in Rust & Python with PySpark-like syntax.

For a long time, Bayesian Hierarchical Modelling has been a very powerful tool that sadly could not be applied often due to its high computations costs. With NumPyro and the latest advances in high-performance computations in Python, Bayesian Hierarchical Modelling is now ready for prime time.

Feature engineering takes up a huge part in the work-life of a data scientist. Sometimes this doesn’t stop at features but also the target variable itself is transformed leading to all kinds of unexpected consequences. In this post, you will learn about common pitfalls, how a transformation can affect the error measure, the math behind it, and even how all this can be used to your advantage.

With the release of Spark 2.3 implementing user defined functions with PySpark became a lot easier and faster. Unfortunately, there are still some rough edges when it comes to complex data types that need to be worked around.