Top 25 Pandas tricks

Here’s a really great tour through some advanced Pandas features, by Kevin Markham of Data School.

Here are the tricks that he features:

  1. Show installed versions
  2. Create an example DataFrame
  3. Rename columns
  4. Reverse row order
  5. Reverse column order
  6. Select columns by data type
  7. Convert strings to numbers
  8. Reduce DataFrame size
  9. Build a DataFrame from multiple files (row-wise)
  10. Build a DataFrame from multiple files (column-wise)
  11. Create a DataFrame from the clipboard
  12. Split a DataFrame into two random subsets
  13. Filter a DataFrame by multiple categories
  14. Filter a DataFrame by largest categories
  15. Handle missing values
  16. Split a string into multiple columns
  17. Expand a Series of lists into a DataFrame
  18. Aggregate by multiple functions
  19. Combine the output of an aggregation with a DataFrame
  20. Select a slice of rows and columns
  21. Reshape a MultiIndexed Series
  22. Create a pivot table
  23. Convert continuous data into categorical data
  24. Change display options
  25. Style a DataFrame
  26. Bonus: Profile a DataFrame

My favorite tip is #25, on styling a dataframe. The bonus tip on Pandas profiling is also pretty cool!

A Jupyter notebook with example usage is available on GitHub.

If you’re hungry for more best practices in Pandas, you can check out Kevin’s PyCon 2019 workshop presentation or his complete series of videos on YouTube.

Setting up PyCharm

I love Sublime Text, and I recently wrote how I optimized it for Python development. But I’ve also admired PyCharm as a full-featured IDE. The problem is that PyCharm is visually cluttered, with buttons, toolbars, and windows everywhere. Certainly, there is a very steep learning curve.

Recently, while I was watching one of Michael Kennedy’s video courses (where the coding examples are done in PyCharm), I was inspired to give PyCharm a closer look.

I was happy to discover that there is a video playlist on YouTube that provides an in-depth Getting Started guide. The JetBrains web site also features a Quick Start guide with really excellent documentation/tutorials.

For scientists especially, be sure to check out IPython/Jupyter Notebook integration in PyCharm.

I plan to spend a lot more time going through this material.