Unified Task Import API: CSV & JSON Bulk Upload

by Admin 48 views
Unified Task Import API: CSV & JSON Bulk Upload

Hey guys! Today, we're diving deep into the exciting world of streamlining task management with a brand new, unified bulk-import endpoint. This is all about making your lives easier when dealing with large datasets, so buckle up!

What's the Big Deal?

So, what's all the hype about a unified bulk-import endpoint? Well, imagine you've got a massive list of tasks you need to get into your system. Before, you might have had to send each task individually, which is slow and painful. Or, you might have had separate processes for different file types. Now, we're introducing a single, streamlined way to upload tasks, whether they're in JSON or CSV format. Think of it as a super-efficient data on-ramp for your task management system!

This new endpoint at POST /tasks/import accepts both application/json and text/csv formats, making it incredibly versatile. This means you can kiss those complicated, multi-step import processes goodbye. Fewer steps, less hassle, more productivity—that's the name of the game.

Why This Matters

Let's break down why this unified approach is a game-changer:

  • Simplifies Client Workflows: No more juggling multiple import methods. Just one endpoint to rule them all!
  • Reduces Round-Trips: Uploading a bunch of tasks at once means fewer requests to the server, which speeds things up significantly.
  • Provides Useful Diagnostics: The endpoint gives you detailed feedback on which tasks were imported successfully and which ones failed, along with the reasons why. No more guessing games!

Diving into the API

Alright, let's get into the nitty-gritty of how this API works. Here’s what you need to know to start using it effectively.

Endpoint Details

  • Endpoint: POST /tasks/import
  • Content-Type: application/json OR text/csv

JSON Body Example

If you're sending your tasks in JSON format, the body should be an array of TaskCreate objects. Each object represents a task and should include the necessary fields, such as title and description.

[
  {"title": "A", "description": "d1"},
  {"title": "B", "description": "d2"}
]

CSV Example

For CSV uploads, the body should be a CSV file with a header row that includes title and description. Make sure your data is properly formatted to avoid any import errors.

title,description
A,d1
B,d2

Success Response

When the import is successful (or partially successful), you'll receive a 201 Created response with a body that summarizes the import results. This includes the number of imported tasks, the number of failed tasks, and details about any errors that occurred.

{
  "imported": 2,
  "failed": 1,
  "errors": [
    {"index": 1, "error": "title must not be empty"}
  ],
  "tasks": [ /* created Task objects with id, timestamps */ ]
}
  • imported: The number of tasks successfully imported.
  • failed: The number of tasks that failed to import.
  • errors: An array of error objects, each containing the index of the failed task and a descriptive error message.
  • tasks: An array of the created Task objects, including their id and timestamps.

Failure Cases

Of course, things don't always go as planned. Here are some common failure scenarios and how the API handles them:

  • Invalid JSON or Invalid UTF-8 CSV: If the JSON is malformed or the CSV contains invalid UTF-8 characters, you'll get a 400 Bad Request error with a helpful error message.
  • Unsupported Content-Type: If you try to send data with an unsupported Content-Type, you'll also get a 400 Bad Request error.

Acceptance Criteria: Ensuring Quality

To make sure this new endpoint is up to snuff, we've defined a set of acceptance criteria. These are the benchmarks we use to ensure everything is working correctly.

  • Single Handler: A single handler import_tasks is implemented in src/handlers/task_handler.rs. This keeps the code organized and maintainable.
  • Router Entry: A router entry POST /tasks/import is wired in src/routes/mod.rs. This ensures the endpoint is accessible.
  • Repository Helper: A repository helper insert_many(&[TaskCreate]) -> Vec<Task> is used to persist valid rows. This makes the database interactions efficient.
  • Task Validation: TaskCreate::validate() (or equivalent) enforces that the title is non-empty. This ensures data integrity.
  • Integration Tests: Comprehensive integration tests are added to cover various scenarios:
    • Fully valid CSV import inserts all rows and returns the correct imported count.
    • A JSON array with one invalid row results in partial success; failed and errors reflect the failure, and valid rows are persisted.
    • CSV with invalid UTF-8 or a completely malformed payload returns a 400 error.
  • README Update: The README is updated with the new endpoint and example responses, making it easy for developers to understand and use the API.

Implementation Notes and Suggestions

Here are some tips and best practices to keep in mind when implementing this feature:

  • Error Messages: Make sure the errors array includes the row/index and a short, descriptive message. This makes it easier for users to understand what went wrong and fix their data.
  • Use Existing Libraries: Leverage the csv crate for parsing CSV and serde_json for JSON parsing. These libraries are well-tested and will save you a lot of time and effort.
  • Server-Side Limits: Consider adding a server-side limit (max rows or max body size) in a follow-up PR to avoid memory issues. This will help ensure the stability of your application.

Suggested Branch and Commit Message

  • Suggested branch name: pr/11-unified-import
  • Suggested commit message: Use the provided commit message above.

Testing and Manual Verification: Putting It to the Test

Before you deploy this new endpoint, it's crucial to test it thoroughly. Here’s how you can do that:

Automated Testing

  • Run cargo test: Make sure all the new integration tests pass. This is your first line of defense against bugs.

Manual Verification

Use curl to manually test the endpoint with different types of data. Here are some examples:

  • JSON:

    curl -X POST http://127.0.0.1:8080/tasks/import \
     -H "Content-Type: application/json" \
     -d '[{"title":"A","description":"x"}]'

  • CSV:

    curl -X POST http://127.0.0.1:8080/tasks/import \
     -H "Content-Type: text/csv" \
     --data-binary {{content}}#39;title,description\nA,x\n'

Wrapping Up

Alright, folks, that's a wrap on the unified bulk-import endpoint! By consolidating the way we handle task imports, we're making it easier than ever to manage large datasets. With support for both JSON and CSV formats, detailed error reporting, and comprehensive testing, this new endpoint is set to become a cornerstone of our task management system. Happy coding, and may your imports always be successful (or at least partially successful with clear error messages)!