Upload CSV to AWS S3 with React/Next.js

Upload CSV to AWS S3 with React: Setup and Code

First, many data apps use sync jobs. Therefore, letting users send CSV files straight to S3 is smart. In addition, this setup splits the UI from background tasks. As a result, direct uploads boost both speed and trust.

Moreover, this guide shows how direct transfers work. Furthermore, we look at common setups and safe rules. Also, you will learn about CORS and React code.

Goals of S3 CSV Uploads

• First, you can upload CSV files safely from the browser.
• Second, tracking upload progress becomes very easy.
• Therefore, coders can show clear errors to users.
• Finally, this setup links smoothly with your backend.
• Consequently, your system can start new tasks right away.

Two Main Setup Models for S3

Model A: Client SDK (AWS SDK v3)

For example, the app uses @aws-sdk/client-s3 and @aws-sdk/lib-storage. Thus, a web app pushes files straight to S3 here. Then, the client gets short-term login tokens from Cognito.

• Overall, this model offers a very simple flow.
• Also, it gives great support for big files.
• However, you must grant IAM rules directly to the client.
• Because of this risk, never put AWS keys in your frontend code.

Model B: Presigned URLs

Conversely, no secret keys live on the client here. Instead, the server makes a quick, temp URL for the file. Next, the frontend simply makes a PUT request to that link.

• Likewise, absolutely no AWS keys exist on the client side.
• But, coders need a backend route to make the URL.
• Besides, you get less control over progress bars.

Ultimately, you should pick a model based on your team. Moreover, it is fine to mix both methods if you want.

Getting Ready for S3 Code

First, make sure your setup is ready before writing code. For instance, you should know a bit about Next.js API Routes. Then, we will use the @aws-sdk/client-s3 tool. In addition, this package is part of the new AWS SDK v3.

Setting Up S3 and CORS

First, make the target bucket for your CSV files. Next, set up CORS rules for the bucket. Therefore, this step allows direct browser uploads. Thus, you will find a small example below:

[ { "AllowedOrigins": ["https://your-frontend.example.com"], "AllowedMethods": ["GET", "PUT", "POST"], "AllowedHeaders": ["*"], "ExposeHeaders": ["ETag", "x-amz-request-id"], "MaxAgeSeconds": 3000 } ]

Setting Up IAM Rules

Overall, you must give users the smallest amount of access.

• For instance, if using the client SDK, grant s3:PutObject only to a narrow folder.
• For example, use uploads/${userId}/*.
• Conversely, if using presigned URLs, only the backend needs access.
• Meanwhile, the client just calls that link.

Method 1: AWS SDK v3 (Client Code)

Installing Packages

First, install the needed packages using npm.

npm i @aws-sdk/client-s3 @aws-sdk/lib-storage

Creating the S3 Client

For example, you can get short-term keys via Cognito. Alternatively, you might use an STS token service. Thus, here is a simple code sketch:

import { S3Client } from "@aws-sdk/client-s3";
// Build the client
export function createS3Client(credentials: { accessKeyId: string; secretAccessKey: string; sessionToken?: string; region: string; }) {
  /* Return instance */
  return new S3Client({ region: credentials.region, credentials: { accessKeyId: credentials.accessKeyId, secretAccessKey: credentials.secretAccessKey, sessionToken: credentials.sessionToken } });
}

Building the React Upload UI

Next, let's build the user interface. Moreover, this component tracks the upload progress well.

import React, { useState } from "react";
// Get upload tool
const { Upload } = require("@aws-sdk/lib-storage");
/* Load S3 types */
import type { PutObjectCommandInput } from "@aws-sdk/client-s3";
// Fetch client
const { createS3Client } = require("./s3Client");

export default function CsvUpload() {
  // Define states
  let [file, setFile] = useState<File | null>(null);
  var [progress, setProgress] = useState<number>(0);
  const [status, setStatus] = useState<string>("");

  async function handleUpload() {
    if (!file) return;

    // Init auth client
    let s3 = createS3Client({ accessKeyId: "AKIA...", secretAccessKey: "SECRET...", sessionToken: "TOKEN...", region: "ap-northeast-1" });

    /* Apply targets */
    var params: PutObjectCommandInput = { Bucket: process.env.NEXT_PUBLIC_AWS_S3_BUCKET!, Key: `uploads/${Date.now()}-${file.name}`, Body: file, ContentType: "text/csv" };

    // Create uploader
    const uploader = new Upload({ client: s3, params, queueSize: 4, partSize: 10 * 1024 * 1024 });

    uploader.on("httpUploadProgress", (evt) => {
      /* Math logic */
      const loaded = evt.loaded ?? 0;
      let total = evt.total ?? file.size;
      setProgress(Math.round((loaded / total) * 100));
    });

    try {
      setStatus("Uploading...");
      await uploader.done();
      setStatus("Done!");
    } catch (e: any) {
      setStatus(`Error: ${e?.message ?? "Upload failed"}`);
    }
  }

  return (
    <div>
      <input type="file" accept=".csv,text/csv" onChange={(e) => setFile(e.target.files?.[0] ?? null)} />
      <button onClick={handleUpload} disabled={!file}>Upload CSV</button>
      <div>Progress: {progress}%</div>
      <div>{status}</div>
    </div>
  );
}

Tips for the SDK

• First, increase queueSize on fast networks.
• On the other hand, drop it for slow clients.
• Second, boost partSize for very big files.
• As a result, you cut down network stress.
• Finally, always set a correct ContentType.
• Thus, you ensure files load right later on.

Method 2: Presigned URLs (No Client Keys)

Making the URL in the Backend

First, you must create a Next.js API route. Next, this route makes the short-lived URL safely.

// pages/api/s3/presign.ts
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
/* Presigner setup */
const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");

export default async function handler(req, res) {
  if (req.method !== "POST") return res.status(405).end();

  // Get body
  let { fileName, contentType } = req.body;

  /* Set region */
  const s3 = new S3Client({ region: "ap-northeast-1" });
  var key = `uploads/${Date.now()}-${fileName}`;

  // Extract url
  let url = await getSignedUrl(s3, new PutObjectCommand({ Bucket: process.env.AWS_S3_BUCKET, Key: key, ContentType: contentType }), { expiresIn: 60 });

  res.status(200).json({ url, key });
}

Uploading from the App

Then, your app requests the URL. Afterward, it uses PUT to send the file to S3.

async function uploadViaPresigned(file: File) {
  // 1) Fetch route
  let res = await fetch("/api/s3/presign", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ fileName: file.name, contentType: "text/csv" }) });
  /* Parse payload */
  var { url, key } = await res.json();

  // 2) Execute S3 action
  const put = await fetch(url, { method: "PUT", body: file, headers: { "Content-Type": "text/csv" } });
  if (!put.ok) throw new Error("Upload failed");

  return { key };
}

• Overall, this plan keeps AWS keys totally off the client.
• However, Presigned URLs do not show native progress.
• Nevertheless, coders can track it using native web tools.

CSV Checks and Prep (Good Idea)

Therefore, we suggest checking the files locally before uploading.

• First, check the MIME type (.csv/text/csv).
• Second, enforce a strict file size limit.
• Also, try reading a few lines with a tool like PapaParse.
• Thus, this helps preview the headers.
• Moreover, set the text encoding to UTF-8.
• In addition, strip out bad BOM characters if needed.
• Finally, use a smart naming rule with timestamps.
• As a result, you easily avoid overwriting old files.

Safety Rules for S3 Data

• First, only allow PutObject on the exact user folder.
• Similarly, use server encryption (SSE-S3) for private data.
• Next, set bucket rules to delete old temp files.
• Also, trigger virus scans via Lambda for extra safety.
• Finally, make sure your server only reads trusted folders.

Common Errors

• For instance, if you see missing headers, check S3 CORS.
• Similarly, a 403 error means your IAM rules are wrong.
• Also, a Signature error comes from the wrong regions.
• Furthermore, files over 5GB truly need multipart uploads.
• So, the SDK v3 supports this right away.
• Finally, if users have bad internet, boost the retries.
• Likewise, reduce the queue size.

UX Tips

• First, show a clear pop-up for big background jobs.
• Second, display a nice progress bar.
• In addition, add a retry button.
• Third, show exact error messages.
• For example, tell users if the size or format is wrong.
• Finally, keep a solid upload log inside your admin panel.

Quick Checklist

• First, set the correct CORS rules for your site.
• Next, pick between a client SDK or a presigned URL.
• Then, force strict security with IAM profiles.
• Furthermore, improve the UI with clear errors.
• Afterward, set up tasks to run right after the files upload.
• Overall, this post gives you two strong ways to handle uploads.

Conclusion

Overall, adding S3 uploads to your app is smart. In fact, it handles massive data without crashing your server. So, moving the file transfer to S3 creates a fast app. As a result, your users enjoy a much better experience.

• Therefore, choose Model A (AWS SDK v3) if you need advanced features. For example, large file uploads and progress bars work best here. Hence, it fits perfectly for internal tools. Also, using Cognito is easy in these cases.

• Alternatively, opt for Model B (Presigned URLs) to keep your app light. In other words, you hide all AWS logic from the user. Ultimately, this remains the top standard for public apps.

However, the upload is just the start. Above all, always add post-upload tasks to build a great system. Consequently, using AWS Lambda ensures your CSV data gets checked fast.

References

• First, review the Image from Gemini.
• Second, examine https://ui.docs.amplify.aws/react/connected-components/storage/fileuploader
• Third, check https://docs.aws.amazon.com/AmazonS3/latest/API/browser-based-uploads-aws-amplify.html