Unit tests should never(❗) call the internet

You can’t always control what your upstream data produce will change

Pytest fixture scope can cause test config side effects

For when you need to break a dataframe up into a bunch of smaller dataframes

Again. . .Spark will hammer a relational DB you connect to if you let it.

I was in a situation a while back where I needed to make an API call to clean up some data in a dataframe. Here is the approach I took.

Spark will hammer the relational DB you connect to if you let it.