Python Tutorial : Transforming DataFrames

Want to learn more? Take the full course at https://learn.datacamp.com/courses/manipulating-dataframes-with-pandas at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work.

---

Once we've selected or filtered our data, we often want to transform it somehow.

The best way to transform data in Pandas DataFrames is with methods inherent to DataFrames.

Next best is using NumPy ufuncs or Universal Functions to transform entire columns of data "elementwise".

Let's see how this works.

Suppose we want to convert sales numbers into units of whole dozens (rounded down) rather than individual item count.

The most efficient way to do this is to use a Pandas built-in method like floordiv.

Notice this arithmetic operation is applied to every entry in the DataFrame without writing a loop.

An other way to do this uses NumPy's floor_divide function.

Both of these strategies use vectorized or elementwise computation to repeat the same computation over the entire data structure without writing a loop.

If Pandas' floordiv and Numpy's floor_divide were not available, we could make a custom function to do this.

We call it dozens here.

The DataFrame apply method called here using dozens executes that function with each entry of the DataFrame (again without writing a loop).

Yet another way to achieve the same result is to use a lambda function with the apply method.

The lambda keyword followed by the input argument, a colon, and the output expression provides a convenient one-line definition of a throwaway function.

All of the preceding computations returned a new DataFrame without altering the original DataFrame df.

To preserve a computed result, we can create a new column storing calculations.

For instance, here, we create a new dozens_of_eggs column in which the floordiv(12) method is applied to the Series df 'eggs'.

Both dot apply and vectorized methods work on Series as well as on entire DataFrames.

Moreover, filters and indices often provide subset Series or DataFrames for transformation.

Having worked with vectorized computation over numerical Series, let's look at string operations next.

Notice the df dot index attribute is itself a special kind of Series containing strings in this instance.

DataFrames, Series, and Index objects come with a handy dot str attribute as a kind of accessor for vectorized string transformations.

Here, we reassign the index using df dot index dot str dot upper to make the index all upperase.

Again, notice we did not loop explicitly over the index entries; we applied a vectorized string method to transform the entire index elementwise.

For the index, there is no apply method.

For the index, the relevant method is called map, an unfortunately different terminology.

Thus, we can apply, say, str dot lower or a custom transformation to the index elementwise using the map method instead.

Many arithmetic operators (for instance the plus sign) work with DataFrames and Series directly.

Thus, here, we create a new column salty_eggs by adding the salt and eggs columns together.

If we an express a calculation using Pandas alone, that's always preferable to using a loop.

That's a lot to absorb.

Take some time to work through the exercises using vectorized computations.

#DataCamp #PythonTutorial #ManipulatingDataFrameswithpandas