MySQL 处理重复数据

预防重复(唯一索引)与清理重复(GROUP BY + 保留最小 id)。

Beginner
Key Points最好的去重是别让重复进来:用唯一索引在写入时拦截。

防止重复写入

SQL
-- 已有重复时先建索引会失败,可忽略:
ALTER IGNORE TABLE users ADD UNIQUE uk_email (email);
Output
Query OK, 0 rows affected (0.05 sec)

清理已有重复

SQL
-- 找出重复的 email
SELECT email, COUNT(*) c FROM users GROUP BY email HAVING c > 1;
-- 保留 id 最小的一条,删掉其余
DELETE u FROM users u
JOIN users d ON d.email = u.email AND d.id < u.id;
Output
Query OK, 2 rows affected (0.02 sec)
ALTER IGNORE changed in 5.6 and is removed in 8.0 — don't rely on it. Create the unique index to surface dupes, clean them, then add the index.

Friends & Partners