Less Than Dot is a community of passionate IT professionals and enthusiasts dedicated to sharing technical knowledge, experience, and assistance. Inside you will find reference materials, interesting technical discussions, and expert tips and commentary. Once you register for an account you will have immediate access to the forums and all past articles and commentaries.
Finding duplicates across columns
From Wiki
You have a table with a bunch of columns and you would like to return all the rows where a combination of columns has duplicate data. So how can you do this? It is pretty easy, you do a group by followed by a count having > 1
First create this table
- CREATE TABLE #Dups(
- id INT IDENTITY,
- Col1 VARCHAR(20),
- Col2 VARCHAR(20),
- Col3 VARCHAR(20),
- Col4 VARCHAR(20),
- Col5 VARCHAR(20),
- SomeOtherCol VARCHAR(20))
Insert this data
- INSERT #Dups VALUES('a','a','a','a','a','a')
- INSERT #Dups VALUES('a','a','a','a','a','a')
- INSERT #Dups VALUES('b','a','a','a','a','a')
- INSERT #Dups VALUES('b','a','a','a','a','a')
- INSERT #Dups VALUES('b','b','b','b','b','b')
- INSERT #Dups VALUES('c','a','a','a','a','a')
- INSERT #Dups VALUES('d','a','a','a','a','a')
- INSERT #Dups VALUES('e','a','a','a','a','a')
- INSERT #Dups VALUES('f','a','a','a','a','a')
- INSERT #Dups VALUES('g','a','a','a','a','a')
- INSERT #Dups VALUES('a','g','a','a','a','a')
- SELECT *
- FROM #Dups
| id | Col1 | Col2 | Col3 | Col4 | Col5 | SomeCol |
| 1 | a | a | a | a | a | a |
| 2 | a | a | a | a | a | a |
| 3 | b | a | a | a | a | a |
| 4 | b | a | a | a | a | a |
| 5 | b | b | b | b | b | b |
| 6 | c | a | a | a | a | a |
| 7 | d | a | a | a | a | a |
| 8 | e | a | a | a | a | a |
| 9 | f | a | a | a | a | a |
| 10 | g | a | a | a | a | a |
| 11 | a | g | a | a | a | a |
Here is the query
- SELECT Col1,Col2,Col3,Col4,Col5,MIN(id),MAX(id)
- FROM #Dups
- GROUP BY Col1,Col2,Col3,Col4,Col5
- HAVING COUNT(*) > 1
| Col1 | Col2 | Col3 | Col4 | Col5 | Min | Max |
| a | a | a | a | a | 1 | 2 |
| b | a | a | a | a | 3 | 4 |
As you can see the duplicate data is displayed
Contributed by: --SQLDenis 03:03, 31 May 2008 (GMT)
Part of SQL Server Programming Hacks
Section Handy tricks
LTD Social Sitings
Note: Watch for social icons on posts by your favorite authors to follow their postings on these and other social sites.