# UTF-8 CSV Imports: Find Where Business Text Becomes Corrupted

<a id="main-content"></a>

Operations &amp; Systems

October 28, 2026  ·  7 min read  ·  Victor Valentine Romo

On this page

1. [Separate the File From Its Display](<#separate-the-file-from-its-display>)
2. [Follow a Synthetic Label Through the Boundary](<#follow-a-synthetic-label-through-the-boundary>)
3. [Use Signatures According to the Receiving Contract](<#use-signatures-according-to-the-receiving-contract>)
4. [Stop When Replacement Characters Appear](<#stop-when-replacement-characters-appear>)
5. [Build an Import Diagnosis Worksheet](<#build-an-import-diagnosis-worksheet>)
6. [Define What the Round Trip Must Preserve](<#define-what-the-round-trip-must-preserve>)
7. [Distinguish a Display Problem From Changed Text](<#distinguish-a-display-problem-from-changed-text>)
8. [Accept the Handoff With Evidence](<#accept-the-handoff-with-evidence>)
9. [Sources](<#sources>)

A product label arrives as `CafÃ©`. Someone corrects the visible cell, saves the [spreadsheet](</articles/spreadsheet-formula-audit-business/>), and calls the import repaired.

Keep the original file before accepting that answer. The next export needs to preserve the intended text too.

[Work backward from the damaged value](</articles/data-lineage-small-business-reports/>) until you find the first boundary where its representation changes. Record that boundary and the setting used there. Test the correction on a copy.

This article proposes a diagnosis worksheet for business imports. The records are fictional. No particular spreadsheet, database, or importer is assumed to use the illustrated settings.

<a id="separate-the-file-from-its-display"></a>

## Separate the File From Its Display

Give four things separate names on the worksheet: stored bytes, decoded characters, visible display, and import configuration.

Bytes are the file evidence. Decoded characters are the text produced when those bytes are interpreted. The display draws that text. Import configuration records the receiver's chosen interpretation.

Ask for evidence at each layer. A screenshot shows a display. It does not establish which bytes arrived or which decoder read them.

Start with a read-only original and a working copy. Record the original [filename](</articles/file-delivery-manifest-business-handoffs/>), receipt date, sender's declared encoding, and file version. Keep later exports under different filenames.

If the sender cannot identify the encoding, write unknown. Treat the filename extension as a file label rather than [an encoding declaration](</articles/api-content-type-response-validation/>).

Use the [CSV data contract](</articles/csv-data-contract-business-handoffs/>) to document the handoff's agreed conditions. Its sample convention belongs to that example. Confirm the convention required by your receiver before applying it elsewhere.

Choose a small set of text values whose intended spelling the data owner can verify. Include characters outside plain English letters. Keep that sample attached to the import decision.

<a id="follow-a-synthetic-label-through-the-boundary"></a>

## Follow a Synthetic Label Through the Boundary

Consider a fictional supplier file containing `Café`, `Piñata`, and `東京`. These labels are test fixtures, not customer records or verified commercial products.

The following results come from encoding each label as UTF-8, then deliberately decoding those same bytes as ISO-8859-1. That wrong-decoder choice is part of the example.

| Expected label | Original UTF-8 bytes, hexadecimal | Wrong ISO-8859-1 decoding |
| --- | --- | --- |
| Café | 43 61 66 C3 A9 | CafÃ© |
| Piñata | 50 69 C3 B1 61 74 61 | PiÃ±ata |
| 東京 | E6 9D B1 E4 BA AC | æ followed by U+009D, then ±äº¬ |

The last row names the control code explicitly because a visible rendering would conceal part of the result. Inspect character values when a screenshot leaves the result ambiguous.

The fixture supplies the expected text. The preserved file supplies the bytes. A separate UTF-8 decoding reproduces the expected labels. The fictional import preview produces the wrong-decoder results.

That evidence locates the first observed change at the preview's decoding boundary. It does not prove that every later stage preserves the corrected text.

For this trial, select UTF-8 in the fictional receiver's explicit decoder field. Reopen the working copy from the preserved original. Do not continue from an already modified preview.

Record the corrected preview, then inspect the stored value after a test import. Export that test record and compare its text again. Keep each observation tied to its own file or record version.

<a id="use-signatures-according-to-the-receiving-contract"></a>

## Use Signatures According to the Receiving Contract

[Unicode](</articles/unicode-normalization-business-record-matching/>)'s [UTF and byte-order-mark FAQ](<https://www.unicode.org/faq/utf_bom.html>) identifies `EF BB BF` as the UTF-8 signature. UTF-8 has no byte-order ambiguity. Its initial BOM signals encoding rather than byte order.

The FAQ also explains that receiving protocols differ. Some require a signature; some receivers do not expect one. Absence of a BOM does not identify an otherwise unknown encoding.

Document the receiver's requirement. Avoid declaring every business CSV correct or incorrect based on BOM presence alone.

For the fictional trial, create two copies from the same verified text: one with the required signature and one without it. Change only that condition. Name the files distinctly and record which receiver setting each trial uses.

If a trial shows an unexpected character at the beginning of the first header, inspect the initial bytes and decoded header. Do not delete characters across the entire file to make one heading look right.

Keep the accepted convention with the file contract. A later operator should be able to select the intended input [without repeating your investigation](</articles/operations-handover-small-team/>).

<a id="stop-when-replacement-characters-appear"></a>

## Stop When Replacement Characters Appear

Unicode's [error-handling explanation](<https://www.unicode.org/faq/utf_bom.html>) describes U+FFFD REPLACEMENT CHARACTER as one possible response to ill-formed input. Other permitted responses include reporting an error or filtering an offending byte.

Preserve the original bytes when `�` appears. Determine whether that character already exists in the source or appears during the observed conversion.

Use a second fictional fixture: the bytes `43 61 66 E9`. Under ISO-8859-1 these represent `Café`. A UTF-8 decoding configured to replace invalid input produces `Caf�` instead.

Saving that damaged result as UTF-8 produces `43 61 66 EF BF BD`. The new file preserves the replacement character. It does not recover the original `E9` byte or establish the intended label.

In this example, recovery starts from the preserved bytes and a verified source declaration. The fixture tells us which decoder fits. A real unknown file requires evidence from its owner or an authoritative export specification.

Do not infer missing letters from surrounding words and call that decoding. If manual correction is necessary, record it as a correction with an owner and evidence.

Mark an unresolved replacement as held. A plausible label is insufficient acceptance evidence when the original spelling remains unknown.

<a id="build-an-import-diagnosis-worksheet"></a>

## Build an Import Diagnosis Worksheet

Use one row per trial. Keep expected and observed samples side by side. The following fields form a proposed operating record, not a Unicode certification checklist.

| Field | What to record |
| --- | --- |
| File version | Original name, receipt, and working-copy identifier |
| Declared encoding | Sender's statement or unknown, with its source |
| Chosen decoder | Exact receiver selection, application version, and trial |
| Signature condition | Observed initial bytes and agreed requirement |
| Expected sample | Owner-confirmed label and explicit code points where useful |
| Observed sample | Preview, stored value, and export observations |
| Round-trip result | Compared artifact, comparison method, and pass or difference |
| Disposition | Accept, hold, or reject, with owner and reason |

Add a short evidence location beside each observation. A screenshot may illustrate the symptom. Keep the underlying file or character sequence available for inspection.

For the `Café` fixture, the declaration is UTF-8. Trial one chooses ISO-8859-1 and shows `CafÃ©`. Its disposition is reject for this input.

Trial two chooses UTF-8 and shows `Café`. Keep its disposition provisional until stored-value and export checks also pass.

Record an application version even when the setting appears obvious. If the receiver exposes no explicit decoder choice, record that limitation. Ask its owner for the documented import path rather than inventing a hidden setting.

Keep one decision per row. A trial that changes the decoder, the file contents, and the import method leaves the cause unresolved.

<a id="define-what-the-round-trip-must-preserve"></a>

## Define What the Round Trip Must Preserve

Choose the comparison before running the trial. Byte equality and text equality answer different questions.

For [a transfer-copy check](</articles/checksum-verification-business-file-handoffs/>), compare the copied file with the preserved original bytes. For an import-export check, compare the intended field values after decoding each artifact using its documented encoding.

Record both results when both matter. In the synthetic signature trial, adding `EF BB BF` changes file bytes. If the receiver recognizes the signature, the product-label text can still match.

Do not treat a changed file hash as proof that the label changed. Do not treat a matching screenshot as proof that file bytes stayed equal.

For the accepted UTF-8 trial, list three [checkpoints](</articles/incremental-api-sync-checkpoint-safety/>): preview equals the fixture, stored field equals the fixture, and exported field equals the fixture. Attach the exact exported file to the worksheet.

Keep [identifier preservation](</articles/excel-preserve-identifiers-csv-import/>) as a separate acceptance condition. Correct accents do not establish that a numeric-looking identifier survived import.

Set the boundary of your conclusion. Three passing sample labels establish the observed sample result under the recorded configuration. They do not establish that every value in an unexamined production file is correct.

<a id="distinguish-a-display-problem-from-changed-text"></a>

## Distinguish a Display Problem From Changed Text

When a character looks wrong, compare its recorded code points with the expected sample before replacing it.

In a fictional second receiver, suppose the stored field contains the expected sequence for `東京`, while one screen shows empty boxes. Keep the stored sequence and the screenshot as separate observations.

Ask the application owner to check font support and its rendering environment. If another supported display draws the same sequence correctly, record that evidence. Avoid changing stored text merely to satisfy one screen.

If the stored sequence itself differs, return to the earlier checkpoints. Find whether the change appears in the preview, after storage, or during export.

Use precise disposition language. “Display unresolved; stored sample matches” tells the next operator what remains. “Encoding fixed” collapses several untested conditions into one claim.

Connect the result to the broader [CRM data-quality review](</articles/crm-data-quality-automation/>) only after the text boundary is understood. Keep representation checks separate from decisions about whether records describe the same entity.

<a id="accept-the-handoff-with-evidence"></a>

## Accept the Handoff With Evidence

Finish the diagnosis with a short decision note. Name the accepted file version, decoder, signature convention, tested samples, and export result.

For the fictional UTF-8 file, the note can state: trial two preserves all three fixture labels through preview, storage, and export. The first trial fails at decoding. The original remains unchanged.

For the replacement-character file, keep the failed UTF-8 trial attached. Accept a corrected trial only after the source owner confirms the encoding and expected label.

If the original is missing, say that the original bytes are unavailable. Request a fresh export. A re-saved damaged file is evidence of the current damage, not proof of its cause.

Give the receiving owner the worksheet and the accepted artifact together. [Record unresolved fields explicitly](</articles/null-blank-zero-business-data/>). Release only the scope that the evidence supports.

Keep the bytes. Name the boundary.

<a id="sources"></a>

## Sources

- [Unicode: UTF-8, UTF-16, UTF-32 and BOM FAQ](<https://www.unicode.org/faq/utf_bom.html>). UTF-8 signatures, receiving-protocol differences, and ill-formed input handling. Checked 2026.10.28. Decoder fixtures are independently calculated synthetic examples. Worksheets and acceptance [procedures](</articles/sop-documentation-b2b/>) are editorial proposals.
