Duplicate SKUs: how to find them and how to merge them

Any product list that has been in use for a year or two contains duplicates. Nobody plans them. They appear one urgent receipt at a time. In daily work they are a nuisance, but during a barcode count they produce the strangest results: one SKU shows 178 pieces too many while its twin shows 180 too few. Cleaning up duplicate SKUs before the count removes a whole class of false gaps.

How duplicates are born

Understanding the causes tells you where to look:

• Two people enter the same product in different ways, for example "Charging cable 1 m" and "USB cable 1m".
• A supplier changes its article number or its packaging, and the new delivery is entered as a new item.
• Colour and size variants are created as separate items, sometimes twice.
• An import from a marketplace or a spreadsheet creates a new item whenever the code does not match exactly, down to a space or a dash.
• A move from Excel to 1C brings every inconsistent spelling along.

These causes share one thing: nobody searched the list before creating the item. Prevention matters more than the clean-up itself.

Five ways to find them

You do not need special software. A spreadsheet export of the item list is enough.

1. Sort by barcode. If the same barcode sits against two items, one of them is almost certainly a duplicate.
2. Normalise the names in a helper column: lower case, no spaces, no hyphens or dots, look-alike Cyrillic letters replaced with Latin ones. Then highlight repeated values in that column.
3. Look for the same supplier article number on two items.
4. Find pairs where one item only has receipts and the other only has sales. The stock has been split between them.
5. Filter negative balances. An item with a negative balance very often has a twin with the matching positive balance.

Treat what you find as suspects, not verdicts. List them for someone who knows the products to confirm one by one: a true duplicate, or two similar products? Items that differ only in colour, size or volume are easy to misjudge.

What a duplicate does to a count

Take the cable that appears twice. SKU A-1042 has a book balance of 300, and A1042 has 180. They are the same product with the same barcode, and 478 pieces are on the shelf. The scanner reads the barcode and assigns every piece to A-1042, the item the barcode is registered to.

The result: A-1042 shows 178 over (478 against 300) and A1042 shows 180 short (0 against 180). Two large gaps, and two people sent off to investigate. Put the two together and the books say 480 against 478 counted. The real difference is two pieces.

Merging without breaking history

1. Choose the master SKU: usually the one with the longest history, the correct barcode and the live marketplace listing.
2. Write the old code into an alias column of the mapping table. Do not simply delete it, because old documents and supplier files still use it.
3. Check open orders, reservations and the balance of both items in every warehouse.
4. Merge in the system, or mark the duplicate inactive and move its balance with a document, depending on what your system allows.
5. Compare the total balance by item before and after the merge. The totals must be identical.

Some pairs are not duplicates at all. If A-2100 is a carton and A-2101 a single piece, with 12 pieces to a carton, do not merge them. Link them in the mapping table with the conversion, so a carton scan counts 12, a piece scan counts 1 and the totals are in pieces.

Keep new duplicates out

Cleaning once is not enough, so build a few habits:

• A naming rule: product type, model, colour, size, in that order, always.
• A rule to search the list before creating a new item.
• One person who approves new items.
• A monthly report of items with the same barcode or the same normalised name.

InventoryHub flags duplicate and look-alike codes while building the code mapping table, and lists them for your confirmation before any scanning begins, so that duplicates are not read as shortages.