w3resource

Pandas: Drop the columns where at least one element is missing


6. Drop Columns with Any Missing Value

Write a Pandas program to drop the columns where at least one element is missing in a given DataFrame.

Test Data:

     ord_no  purch_amt    ord_date  customer_id  salesman_id
0   70001.0     150.50  2012-10-05         3002       5002.0
1       NaN     270.65  2012-09-10         3001       5003.0
2   70002.0      65.26         NaN         3001       5001.0
3   70004.0     110.50  2012-08-17         3003          NaN
4       NaN     948.50  2012-09-10         3002       5002.0
5   70005.0    2400.60  2012-07-27         3001       5001.0
6       NaN    5760.00  2012-09-10         3001       5001.0
7   70010.0    1983.43  2012-10-10         3004          NaN
8   70003.0    2480.40  2012-10-10         3003       5003.0
9   70012.0     250.45  2012-06-27         3002       5002.0
10      NaN      75.29  2012-08-17         3001       5003.0
11  70013.0    3045.60  2012-04-25         3001          NaN

Sample Solution:

Python Code :

import pandas as pd
import numpy as np
pd.set_option('display.max_rows', None)
#pd.set_option('display.max_columns', None)
df = pd.DataFrame({
'ord_no':[70001,np.nan,70002,70004,np.nan,70005,np.nan,70010,70003,70012,np.nan,70013],
'purch_amt':[150.5,270.65,65.26,110.5,948.5,2400.6,5760,1983.43,2480.4,250.45, 75.29,3045.6],
'ord_date': ['2012-10-05','2012-09-10',np.nan,'2012-08-17','2012-09-10','2012-07-27','2012-09-10','2012-10-10','2012-10-10','2012-06-27','2012-08-17','2012-04-25'],
'customer_id':[3002,3001,3001,3003,3002,3001,3001,3004,3003,3002,3001,3001],
'salesman_id':[5002,5003,5001,np.nan,5002,5001,5001,np.nan,5003,5002,5003,np.nan]})

print("Original Orders DataFrame:")
print(df)
print("\nDrop the columns where at least one element is missing:")
result = df.dropna(axis='columns')
print(result)

Sample Output:

Original Orders DataFrame:
     ord_no  purch_amt    ord_date  customer_id  salesman_id
0   70001.0     150.50  2012-10-05         3002       5002.0
1       NaN     270.65  2012-09-10         3001       5003.0
2   70002.0      65.26         NaN         3001       5001.0
3   70004.0     110.50  2012-08-17         3003          NaN
4       NaN     948.50  2012-09-10         3002       5002.0
5   70005.0    2400.60  2012-07-27         3001       5001.0
6       NaN    5760.00  2012-09-10         3001       5001.0
7   70010.0    1983.43  2012-10-10         3004          NaN
8   70003.0    2480.40  2012-10-10         3003       5003.0
9   70012.0     250.45  2012-06-27         3002       5002.0
10      NaN      75.29  2012-08-17         3001       5003.0
11  70013.0    3045.60  2012-04-25         3001          NaN

Drop the columns where at least one element is missing:
    purch_amt  customer_id
0      150.50         3002
1      270.65         3001
2       65.26         3001
3      110.50         3003
4      948.50         3002
5     2400.60         3001
6     5760.00         3001
7     1983.43         3004
8     2480.40         3003
9      250.45         3002
10      75.29         3001
11    3045.60         3001

For more Practice: Solve these Related Problems:

  • Write a Pandas program to drop columns that contain any missing values from a DataFrame.
  • Write a Pandas program to remove all columns with NaN entries using dropna(axis=1) and display the remaining columns.
  • Write a Pandas program to filter out DataFrame columns that have at least one NaN and then list the cleaned column names.
  • Write a Pandas program to apply dropna() along the columns axis and then compare the shape of the DataFrame before and after.

Go to:


Previous: Write a Pandas program to drop the rows where at least one element is missing in a given DataFrame.
Next: Write a Pandas program to drop the rows where all elements are missing in a given DataFrame.

Python Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

What is the difficulty level of this exercise?

Test your Programming skills with w3resource's quiz.



Follow us on Facebook and Twitter for latest update.