w3resource

Pandas: Drop the columns where at least one element is missing

Pandas Handling Missing Values: Exercise-6 with Solution

Write a Pandas program to drop the columns where at least one element is missing in a given DataFrame.

Test Data:

     ord_no  purch_amt    ord_date  customer_id  salesman_id
0   70001.0     150.50  2012-10-05         3002       5002.0
1       NaN     270.65  2012-09-10         3001       5003.0
2   70002.0      65.26         NaN         3001       5001.0
3   70004.0     110.50  2012-08-17         3003          NaN
4       NaN     948.50  2012-09-10         3002       5002.0
5   70005.0    2400.60  2012-07-27         3001       5001.0
6       NaN    5760.00  2012-09-10         3001       5001.0
7   70010.0    1983.43  2012-10-10         3004          NaN
8   70003.0    2480.40  2012-10-10         3003       5003.0
9   70012.0     250.45  2012-06-27         3002       5002.0
10      NaN      75.29  2012-08-17         3001       5003.0
11  70013.0    3045.60  2012-04-25         3001          NaN

Sample Solution:

Python Code :

import pandas as pd
import numpy as np
pd.set_option('display.max_rows', None)
#pd.set_option('display.max_columns', None)
df = pd.DataFrame({
'ord_no':[70001,np.nan,70002,70004,np.nan,70005,np.nan,70010,70003,70012,np.nan,70013],
'purch_amt':[150.5,270.65,65.26,110.5,948.5,2400.6,5760,1983.43,2480.4,250.45, 75.29,3045.6],
'ord_date': ['2012-10-05','2012-09-10',np.nan,'2012-08-17','2012-09-10','2012-07-27','2012-09-10','2012-10-10','2012-10-10','2012-06-27','2012-08-17','2012-04-25'],
'customer_id':[3002,3001,3001,3003,3002,3001,3001,3004,3003,3002,3001,3001],
'salesman_id':[5002,5003,5001,np.nan,5002,5001,5001,np.nan,5003,5002,5003,np.nan]})

print("Original Orders DataFrame:")
print(df)
print("\nDrop the columns where at least one element is missing:")
result = df.dropna(axis='columns')
print(result)

Sample Output:

Original Orders DataFrame:
     ord_no  purch_amt    ord_date  customer_id  salesman_id
0   70001.0     150.50  2012-10-05         3002       5002.0
1       NaN     270.65  2012-09-10         3001       5003.0
2   70002.0      65.26         NaN         3001       5001.0
3   70004.0     110.50  2012-08-17         3003          NaN
4       NaN     948.50  2012-09-10         3002       5002.0
5   70005.0    2400.60  2012-07-27         3001       5001.0
6       NaN    5760.00  2012-09-10         3001       5001.0
7   70010.0    1983.43  2012-10-10         3004          NaN
8   70003.0    2480.40  2012-10-10         3003       5003.0
9   70012.0     250.45  2012-06-27         3002       5002.0
10      NaN      75.29  2012-08-17         3001       5003.0
11  70013.0    3045.60  2012-04-25         3001          NaN

Drop the columns where at least one element is missing:
    purch_amt  customer_id
0      150.50         3002
1      270.65         3001
2       65.26         3001
3      110.50         3003
4      948.50         3002
5     2400.60         3001
6     5760.00         3001
7     1983.43         3004
8     2480.40         3003
9      250.45         3002
10      75.29         3001
11    3045.60         3001

Python Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Write a Pandas program to drop the rows where at least one element is missing in a given DataFrame.
Next: Write a Pandas program to drop the rows where all elements are missing in a given DataFrame.

What is the difficulty level of this exercise?

Test your Programming skills with w3resource's quiz.



Follow us on Facebook and Twitter for latest update.

Python: Tips of the Day

Creates a list of elements, grouped based on the position in the original lists:

Example:

def tips_zip(*args, fill_value=None):
  	max_length = max([len(lst) for lst in args])
 	 result = []
  	for i in range(max_length):
   	 result.append([
      args[k][i] if i < len(args[k]) else fillvalue for k in range(len(args))
   	 ])
 	 return result
print(tips_zip(['a', 'b'], [1, 2], [True, False]))
print(tips_zip(['a'], [1, 2], [True, False]))
print(tips_zip(['a'], [1, 2], [True, False], fill_value = '1'))

Output:

[['a', 1, True], ['b', 2, False]]
[['a', 1, True], [None, 2, False]]
[['a', 1, True], ['1', 2, False]]