Here’s a really great tour through some advanced Pandas features, by Kevin Markham of Data School.
Here are the tricks that he features:
- Show installed versions
- Create an example DataFrame
- Rename columns
- Reverse row order
- Reverse column order
- Select columns by data type
- Convert strings to numbers
- Reduce DataFrame size
- Build a DataFrame from multiple files (row-wise)
- Build a DataFrame from multiple files (column-wise)
- Create a DataFrame from the clipboard
- Split a DataFrame into two random subsets
- Filter a DataFrame by multiple categories
- Filter a DataFrame by largest categories
- Handle missing values
- Split a string into multiple columns
- Expand a Series of lists into a DataFrame
- Aggregate by multiple functions
- Combine the output of an aggregation with a DataFrame
- Select a slice of rows and columns
- Reshape a MultiIndexed Series
- Create a pivot table
- Convert continuous data into categorical data
- Change display options
- Style a DataFrame
- Bonus: Profile a DataFrame
My favorite tip is #25, on styling a dataframe. The bonus tip on Pandas profiling is also pretty cool!
A Jupyter notebook with example usage is available on GitHub.
If you’re hungry for more best practices in Pandas, you can check out Kevin’s PyCon 2019 workshop presentation or his complete series of videos on YouTube.
I love Sublime Text, and I recently wrote how I optimized it for Python development. But I’ve also admired PyCharm as a full-featured IDE. The problem is that PyCharm is visually cluttered, with buttons, toolbars, and windows everywhere. Certainly, there is a very steep learning curve.
Recently, while I was watching one of Michael Kennedy’s video courses (where the coding examples are done in PyCharm), I was inspired to give PyCharm a closer look.
I was happy to discover that there is a video playlist on YouTube that provides an in-depth Getting Started guide. The JetBrains web site also features a Quick Start guide with really excellent documentation/tutorials.
For scientists especially, be sure to check out IPython/Jupyter Notebook integration in PyCharm.
I plan to spend a lot more time going through this material.
Ever wonder how to read, parse, and write CSV files in Python? This video tutorial from Corey Schafer has the answers.
I received a good recommendation on PythonistaCafe today to check out this video by Corey Schafer: CSV Module – How to Read, Parse, and Write CSV Files.