MySQL 处理重复数据
预防重复(唯一索引)与清理重复(GROUP BY + 保留最小 id)。
Key Points最好的去重是别让重复进来:用唯一索引在写入时拦截。
防止重复写入
SQL
-- 已有重复时先建索引会失败,可忽略:
ALTER IGNORE TABLE users ADD UNIQUE uk_email (email);Output
Query OK, 0 rows affected (0.05 sec)清理已有重复
SQL
-- 找出重复的 email
SELECT email, COUNT(*) c FROM users GROUP BY email HAVING c > 1;
-- 保留 id 最小的一条,删掉其余
DELETE u FROM users u
JOIN users d ON d.email = u.email AND d.id < u.id;Output
Query OK, 2 rows affected (0.02 sec)ALTER IGNORE changed in 5.6 and is removed in 8.0 — don't rely on it. Create the unique index to surface dupes, clean them, then add the index.防止重复写入
SQL
-- 已有重复时先建索引会失败,可忽略:
ALTER IGNORE TABLE users ADD UNIQUE uk_email (email);Output
Query OK, 0 rows affected (0.05 sec)清理已有重复
SQL
-- 找出重复的 email
SELECT email, COUNT(*) c FROM users GROUP BY email HAVING c > 1;
-- 保留 id 最小的一条,删掉其余
DELETE u FROM users u
JOIN users d ON d.email = u.email AND d.id < u.id;Output
Query OK, 2 rows affected (0.02 sec)ALTER IGNORE 在 5.6 后行为有变、8.0 已移除,别依赖它。更稳的做法:先 CREATE UNIQUE INDEX 抓重复行报错,再用上面的 DELETE 清理后再建索引。