Use Django, Celery, and Redis to Automate CSV Data Import
IT Support Technician with 4 years of experience in the IT field as an IT Support Technician, DevOps, and Software Developer. Currently, I am specializing in Cloud and DevOps technologies, by working on projects based on real-world scenarios, with a focus on services and technologies from AWS, Microsoft Azure, Google Cloud, and Oracle Cloud, intending to help companies migrate from On-Premises Environment to the Cloud.
In this small project, I Used Django, a Python Framework to build a robust application that imports any large CSV Dataset into a Table. The app allows you to specify which model you want to import data into, the application will check the header of the CSV files to see if that matches the fields of the model you choose, and if they match data will be imported into that table and if not the application will throw an error message of what happened.
Importing a dataset with 50 records will take a few seconds, but importing a Dataset with millions of records will take minutes, and you don't want users of your application to wait for that time to get a response. That's why I used Celery and Redis to execute that task in the background and notify the user by email once all the dataset records were imported into the table.
Functionality
Import CSV Dataset and specify Table
Display Error/Success Messages
Send an Email when all records are imported
Git hub project at GitHub repo
