Sequential Python Dictionary Transformations with Functional Programming
84. Dictionary Transformation Pipeline
Write a Python function that takes a list of dictionaries and a series of transformation functions, then applies each transformation in sequence to the dictionary list.
Each transformation function should take a list of dictionaries and return a modified list.
Solution:
Python Code:
# Define a function to apply a series of transformation functions to a list of dictionaries.
def transform_dicts(dict_list, transformations):
"""
Applies a series of transformation functions to a list of dictionaries.
Args:
dict_list: List of dictionaries to transform
transformations: List of functions that each take a list of dictionaries
and return a modified list
Returns:
Transformed list of dictionaries
"""
# Initialize the result with the original list of dictionaries.
result = dict_list
# Iterate through each transformation function in the transformations list.
for transform_func in transformations:
# Apply the current transformation function to the result and update it.
result = transform_func(result)
# Return the final transformed list of dictionaries.
return result
# Example data for transformation pipeline
# A list of user dictionaries containing details like name, age, active status, and email.
user_data = [
{'name': 'Nuha', 'age': 30, 'active': True, 'email': '[email protected]'},
{'name': 'Feride', 'age': 25, 'active': False, 'email': '[email protected]'},
{'name': 'Bob', 'age': 40, 'active': True, 'email': '[email protected]'},
{'name': 'Dagmar', 'age': 35, 'active': True, 'email': '[email protected]'},
{'name': 'Marissa', 'age': 22, 'active': False, 'email': '[email protected]'}
]
# Example transformation functions
# Filters out users who are not active (active=False).
def filter_active_users(users):
return [user for user in users if user['active']]
# Adds a new field 'is_adult' to each user dictionary based on their age (>= 18).
def add_is_adult_field(users):
return [{**user, 'is_adult': user['age'] >= 18} for user in users]
# Sorts the list of users by their age in ascending order.
def sort_by_age(users):
return sorted(users, key=lambda x: x['age'])
# Adds a new field 'email_domain' to each user dictionary, extracting the domain from their email.
def format_emails(users):
return [{**user, 'email_domain': user['email'].split('@')[1]} for user in users]
# Define the list of transformations to apply in sequence.
transformations = [
filter_active_users, # Step 1: Filter out inactive users.
add_is_adult_field, # Step 2: Add 'is_adult' field based on age.
sort_by_age, # Step 3: Sort users by age.
format_emails # Step 4: Add 'email_domain' field from email.
]
# Apply the transformations to the user_data using the transform_dicts function.
result = transform_dicts(user_data, transformations)
# Print the final transformed list of dictionaries.
print(result)
Output:
[{'name': 'Nuha', 'age': 30, 'active': True, 'email': '[email protected]', 'is_adult': True, 'email_domain': 'example.com'}, {'name': 'Dagmar', 'age': 35, 'active': True, 'email': '[email protected]', 'is_adult': True, 'email_domain': 'example.com'}, {'name': 'Bob', 'age': 40, 'active': True, 'email': '[email protected]', 'is_adult': True, 'email_domain': 'example.com'}]
Explanation of Each Line:
- Function Definition : Defines transform_dicts, a function to apply a series of transformations to a list of dictionaries.
- Docstring : Provides a description of the function, its arguments, and its return value.
- Initialize Result : Starts with the original list of dictionaries (dict_list) as the initial result.
- Iterate Transformations : Loops through each transformation function in the transformations list.
- Apply Transformation : Applies the current transformation function to the result and updates it.
- Return Result : Returns the final transformed list of dictionaries after all transformations are applied.
- Example Data : Defines a sample list of user dictionaries with fields like name, age, active status, and email.
- Filter Active Users : A transformation function that filters out inactive users (active=False).
- Add Adult Field : A transformation function that adds an is_adult field to each user dictionary based on their age.
- Sort by Age : A transformation function that sorts the list of users by their age in ascending order.
- Format Emails : A transformation function that extracts the email domain and adds it as a new field (email_domain).
- Define Transformations : Creates a list of transformation functions to be applied in sequence.
- Apply Transformations : Calls transform_dicts with the user data and the list of transformations.
- Print Result : Prints the final transformed list of dictionaries to verify the output.
Explanation - Dictionary Transformation Pipeline
- Concept: Apply a series of transformation functions to a list of dictionaries in sequence.
- Challenge: Design a flexible system for chaining multiple data transformations.
- Key Skills:
- Functional programming patterns
- Function composition
- Data pipeline design
- Applications:
- Data cleaning and preprocessing
- ETL (Extract, Transform, Load) processes
- Report generation from raw data
- Feature engineering in machine learning
- Benefits:
- Creates modular, maintainable data processing code
- Allows each transformation to focus on a single responsibility
- Makes complex data processing workflows easier to understand
For more Practice: Solve these Related Problems:
- Write a Python function to apply a sequence of mapping transformations on a dictionary-based dataset.
- Write a Python function that takes a list of dictionaries and applies different transformations based on conditional rules.
- Write a Python function that dynamically modifies dictionary values based on external configuration files.
- Write a Python function to normalize a list of dictionaries by ensuring they all have the same set of keys.
Go to:
Previous: Custom Dictionary with Default Factory.
Next: Dictionary-based Graph Algorithms.
Python Code Editor:
What is the difficulty level of this exercise?
Test your Programming skills with w3resource's quiz.