![Page 1: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/1.jpg)
Lecture 13 - Fei-Fei Li
Lecture 13: k-‐means and mean-‐shi4 clustering
Juan Carlos Niebles Stanford AI Lab
Professor Fei-‐Fei Li Stanford Vision Lab
3-‐Nov-‐15 1
![Page 2: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/2.jpg)
Lecture 13 - Fei-Fei Li
Recap: Gestalt Theory • Gestalt: whole or group
– Whole is greater than sum of its parts – RelaLonships among parts can yield new properLes/features
• Psychologists idenLfied series of factors that predispose set of elements to be grouped (by human visual system)
Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung, Vol. 4, pp. 301-350, 1923 http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm
“I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.”
Max Wertheimer (1880-1943)
3-‐Nov-‐15 2
![Page 3: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/3.jpg)
Lecture 13 - Fei-Fei Li
Recap: Gestalt Factors
• These factors make intuiLve sense, but are very difficult to translate into algorithms.
3-‐Nov-‐15 3
![Page 4: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/4.jpg)
Lecture 13 - Fei-Fei Li
Recap: Image SegmentaLon
• Goal: idenLfy groups of pixels that go together
3-‐Nov-‐15 4
![Page 5: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/5.jpg)
Lecture 13 - Fei-Fei Li
What will we learn today?
• K-‐means clustering • Mean-‐shi4 clustering
3-‐Nov-‐15 5
Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shi4: A Robust Approach toward Feature Space Analysis, PAMI 2002.
![Page 6: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/6.jpg)
Lecture 13 - Fei-Fei Li
What will we learn today?
• K-‐means clustering • Mean-‐shi4 clustering
3-‐Nov-‐15 6
Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shi4: A Robust Approach toward Feature Space Analysis, PAMI 2002.
![Page 7: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/7.jpg)
Lecture 13 - Fei-Fei Li
Image SegmentaLon: Toy Example
• These intensiLes define the three groups. • We could label every pixel in the image according to which
of these primary intensiLes it is. – i.e., segment the image based on the intensity feature.
• What if the image isn’t quite so simple?
intensity input image
black pixels gray pixels
white pixels
1 2 3
Slide credit: Kristen Grauman
3-‐Nov-‐15 7
![Page 8: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/8.jpg)
Lecture 13 - Fei-Fei Li
Pixel cou
nt
Input image
Input image Intensity
Pixel cou
nt
Intensity
Slide credit: Kristen Grauman
3-‐Nov-‐15 8
![Page 9: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/9.jpg)
Lecture 13 - Fei-Fei Li
• Now how to determine the three main intensiLes that define our groups?
• We need to cluster.
Input image Intensity
Pixel cou
nt
Slide credit: Kristen Grauman
3-‐Nov-‐15 9
![Page 10: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/10.jpg)
Lecture 13 - Fei-Fei Li
• Goal: choose three “centers” as the representaLve intensiLes, and label every pixel according to which of these centers it is nearest to.
• Best cluster centers are those that minimize Sum of Square Distance (SSD) between all points and their nearest cluster center ci:
Slid
e cr
edit
: Kr
iste
n G
raum
an
0 190 255
1 2 3
Intensity
3-‐Nov-‐15 10
SSD = x − ci( )2x∈clusteri∑
clusteri∑
![Page 11: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/11.jpg)
Lecture 13 - Fei-Fei Li
Clustering for SummarizaLon
Goal: cluster to minimize variance in data given clusters – Preserve informaLon
Whether is assigned to
Cluster center Data
Slide: Derek Hoiem
c*, δ* = argminc, δ
1N
δiji
K
∑ ci − x j( )2
j
N
∑
x j ci
3-‐Nov-‐15 11
![Page 12: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/12.jpg)
Lecture 13 - Fei-Fei Li
Clustering • With this objecLve, it is a “chicken and egg” problem: – If we knew the cluster centers, we could allocate points to groups by assigning each to its closest center.
– If we knew the group memberships, we could get the centers by compuLng the mean per group.
Slid
e cr
edit
: Kr
iste
n G
raum
an
3-‐Nov-‐15 12
![Page 13: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/13.jpg)
Lecture 13 - Fei-Fei Li
K-‐means clustering 1. IniLalize ( ): cluster centers
2. Compute : assign each point to the closest center – denotes the set of assignment for each to cluster at iteraLon t
3. Computer : update cluster centers as the mean of the points
4. Update , Repeat Step 2-‐3 Lll stopped
Slide: Derek Hoiem
c1,...,cK
δ t = argminδ
1N
δ t−1iji
K
∑ ct−1i − x j( )2
j
N
∑
ct = argminc
1N
δ tiji
K
∑ ct−1i − x j( )2
j
N
∑
t = t +1
technical
note
x j ciδ t
t = 0
δ t
ct
3-‐Nov-‐15 13
![Page 14: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/14.jpg)
Lecture 13 - Fei-Fei Li
K-‐means clustering 1. IniLalize ( ): cluster centers
2. Compute : assign each point to the closest center
3. Computer : update cluster centers as the mean of the points
4. Update , Repeat Step 2-‐3 Lll stopped
Slide: Derek Hoiem
c1,...,cKtechn
ical
note t = 0
δ t
ct
• Commonly used: random iniLalizaLon • Or greedily choose K to minimize residual
• Typical distance measure: • Euclidean • Cosine • Others
• doesn’t change anymore. ct
sim(x, !x ) = xT !xsim(x, !x ) = xT !x x ⋅ !x( )
t = t +1
ct = argminc
1N
δ tiji
K
∑ ct−1i − x j( )2
j
N
∑
3-‐Nov-‐15 14
![Page 15: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/15.jpg)
Lecture 13 - Fei-Fei Li
K-‐means clustering
Illustration Source: wikipedia
1. Initialize Cluster Centers
2. Assign Points to Clusters
3. Re-compute Means
Repeat (2) and (3)
• Java demo: hmp://home.dei.polimi.it/mameucc/Clustering/tutorial_html/AppletKM.html
3-‐Nov-‐15 15
![Page 16: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/16.jpg)
Lecture 13 - Fei-Fei Li
• Converges to a local minimum soluLon – IniLalize mulLple runs
• Bemer fit for spherical data
• Need to pick K (# of clusters)
3-‐Nov-‐15 16
K-‐means clustering techn
ical
note
![Page 17: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/17.jpg)
Lecture 13 - Fei-Fei Li
SegmentaLon as Clustering
K=2
K=3 img_as_col = double(im(:)); cluster_membs = kmeans(img_as_col, K); labelim = zeros(size(im)); for i=1:k inds = find(cluster_membs==i); meanval = mean(img_as_column(inds)); labelim(inds) = meanval; end
Slide credit: Kristen Grauman
3-‐Nov-‐15 17
![Page 18: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/18.jpg)
Lecture 13 - Fei-Fei Li
K-‐Means++
• Can we prevent arbitrarily bad local minima?
1. Randomly choose first center. 2. Pick new center with prob. proporLonal to
– (ContribuLon of x to total error) 3. Repeat unLl K centers.
• Expected error * opLmal
Arthur & Vassilvitskii 2007
Slid
e cr
edit
: St
eve
Seit
z
3-‐Nov-‐15 18
x − ci( )2
=O log K( )
![Page 19: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/19.jpg)
Lecture 13 - Fei-Fei Li
Feature Space • Depending on what we choose as the feature space, we can group pixels in different ways.
• Grouping pixels based on intensity similarity
• Feature space: intensity value (1D) Slide credit: Kristen Grauman
3-‐Nov-‐15 19
![Page 20: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/20.jpg)
Lecture 13 - Fei-Fei Li
Feature Space • Depending on what we choose as the feature space, we can
group pixels in different ways.
• Grouping pixels based on color similarity
• Feature space: color value (3D)
R=255 G=200 B=250
R=245 G=220 B=248
R=15 G=189 B=2
R=3 G=12 B=2
R
G B
Slide credit: Kristen Grauman
3-‐Nov-‐15 20
![Page 21: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/21.jpg)
Lecture 13 - Fei-Fei Li
Feature Space • Depending on what we choose as the feature space, we can
group pixels in different ways.
• Grouping pixels based on texture similarity
• Feature space: filter bank responses (e.g., 24D)
Filter bank of 24 filters
F24
F2
F1
…
Slide credit: Kristen Grauman
3-‐Nov-‐15 21
![Page 22: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/22.jpg)
Lecture 13 - Fei-Fei Li
Smoothing Out Cluster Assignments
• Assigning a cluster label per pixel may yield outliers:
• How can we ensure they are spaLally smooth? 1 2
3 ?
Original Labeled by cluster center’s intensity
Slide credit: Kristen Grauman
3-‐Nov-‐15 22
![Page 23: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/23.jpg)
Lecture 13 - Fei-Fei Li
SegmentaLon as Clustering • Depending on what we choose as the feature space, we can group pixels in different ways.
• Grouping pixels based on intensity+posi9on similarity
⇒ Way to encode both similarity and proximity. Slide credit: Kristen Grauman
X
Intensity
Y
3-‐Nov-‐15 23
![Page 24: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/24.jpg)
Lecture 13 - Fei-Fei Li
K-‐Means Clustering Results • K-‐means clustering based on intensity or color is essenLally vector quanLzaLon of the image amributes – Clusters don’t have to be spaLally coherent
Image Intensity-‐based clusters Color-‐based clusters
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
3-‐Nov-‐15 24
![Page 25: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/25.jpg)
Lecture 13 - Fei-Fei Li
K-‐Means Clustering Results
• K-‐means clustering based on intensity or color is essenLally vector quanLzaLon of the image amributes – Clusters don’t have to be spaLally coherent
• Clustering based on (r,g,b,x,y) values enforces more spaLal coherence
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
3-‐Nov-‐15 25
![Page 26: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/26.jpg)
Lecture 13 - Fei-Fei Li
How to evaluate clusters?
• GeneraLve – How well are points reconstructed from the clusters?
• DiscriminaLve – How well do the clusters correspond to labels?
• Purity – Note: unsupervised clustering does not aim to be discriminaLve
Slide: Derek Hoiem 3-‐Nov-‐15 26
![Page 27: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/27.jpg)
Lecture 13 - Fei-Fei Li
How to choose the number of clusters?
• ValidaLon set – Try different numbers of clusters and look at performance
• When building dicLonaries (discussed later), more clusters typically work bemer
Slide: Derek Hoiem 3-‐Nov-‐15 27
![Page 28: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/28.jpg)
Lecture 13 - Fei-Fei Li
K-‐Means pros and cons • Pros
• Finds cluster centers that minimize condiLonal variance (good representaLon of data)
• Simple and fast, Easy to implement • Cons
• Need to choose K • SensiLve to outliers • Prone to local minima • All clusters have the same parameters
(e.g., distance measure is non-‐adapLve)
• *Can be slow: each iteraLon is O(KNd) for N d-‐dimensional points
• Usage • Unsupervised clustering • Rarely used for pixel segmentaLon
3-‐Nov-‐15 28
![Page 29: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/29.jpg)
Lecture 13 - Fei-Fei Li
What will we learn today?
• K-‐means clustering • Mean-‐shi4 clustering
3-‐Nov-‐15 29
Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shi4: A Robust Approach toward Feature Space Analysis, PAMI 2002.
![Page 30: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/30.jpg)
Lecture 13 - Fei-Fei Li
Mean-‐Shi4 SegmentaLon
• An advanced and versaLle technique for clustering-‐based segmentaLon
hmp://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
D. Comaniciu and P. Meer, Mean Shi4: A Robust Approach toward Feature Space Analysis, PAMI 2002. Slid
e cr
edit
: Sv
etla
na L
azeb
nik
3-‐Nov-‐15 30
![Page 31: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/31.jpg)
Lecture 13 - Fei-Fei Li
Mean-‐Shi4 Algorithm
• IteraLve Mode Search 1. IniLalize random seed, and window W 2. Calculate center of gravity (the “mean”) of W: 3. Shi4 the search window to the mean 4. Repeat Step 2 unLl convergence
Slid
e cr
edit
: St
eve
Seit
z
3-‐Nov-‐15 31
![Page 32: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/32.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 32
![Page 33: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/33.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 33
![Page 34: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/34.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 34
![Page 35: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/35.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 35
![Page 36: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/36.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 36
![Page 37: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/37.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean Shift vector
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 37
![Page 38: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/38.jpg)
Lecture 13 - Fei-Fei Li
Region of interest
Center of mass
Mean-‐Shi4
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 38
![Page 39: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/39.jpg)
Lecture 13 - Fei-Fei Li
Tessellate the space with windows Run the procedure in parallel Slide by Y. U
krainitz & B. Sarel
Real Modality Analysis
3-‐Nov-‐15 39
![Page 40: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/40.jpg)
Lecture 13 - Fei-Fei Li
The blue data points were traversed by the windows towards the mode. Slide by Y. U
krainitz & B. Sarel
Real Modality Analysis
3-‐Nov-‐15 40
![Page 41: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/41.jpg)
Lecture 13 - Fei-Fei Li
Mean-‐Shi4 Clustering
• Cluster: all data points in the amracLon basin of a mode
• AmracLon basin: the region for which all trajectories lead to the same mode
Slide by Y. Ukrainitz & B. Sarel
3-‐Nov-‐15 41
![Page 42: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/42.jpg)
Lecture 13 - Fei-Fei Li
Mean-‐Shi4 Clustering/SegmentaLon • Find features (color, gradients, texture, etc) • IniLalize windows at individual pixel locaLons • Perform mean shi4 for each window unLl convergence • Merge windows that end up near the same “peak” or mode
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
3-‐Nov-‐15 42
![Page 43: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/43.jpg)
Lecture 13 - Fei-Fei Li
Mean-‐Shi4 SegmentaLon Results
hmp://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html Slid
e cr
edit
: Sv
etla
na L
azeb
nik
3-‐Nov-‐15 43
![Page 44: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/44.jpg)
Lecture 13 - Fei-Fei Li
More Results
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
3-‐Nov-‐15 44
![Page 45: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/45.jpg)
Lecture 13 - Fei-Fei Li
More Results
3-‐Nov-‐15 45
![Page 46: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/46.jpg)
Lecture 13 - Fei-Fei Li
• Need to shi4 many windows… • Many computaLons will be redundant.
Problem: ComputaLonal Complexity
Slid
e cr
edit
: Ba
stia
n Le
ibe
3-‐Nov-‐15 46
![Page 47: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/47.jpg)
Lecture 13 - Fei-Fei Li
Speedups: Basin of AmracLon
r
Slid
e cr
edit
: Ba
stia
n Le
ibe
3-‐Nov-‐15 47
1. Assign all points within radius r of end point to the mode.
![Page 48: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/48.jpg)
Lecture 13 - Fei-Fei Li
Speedups
r =c
Slid
e cr
edit
: Ba
stia
n Le
ibe
3-‐Nov-‐15 48
2. Assign all points within radius r/c of the search path to the mode -‐> reduce the number of data points to search.
![Page 49: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/49.jpg)
Lecture 13 - Fei-Fei Li
Summary Mean-‐Shi4 • Pros
– General, applicaLon-‐independent tool – Model-‐free, does not assume any prior shape (spherical, ellipLcal, etc.) on data clusters
– Just a single parameter (window size h) • h has a physical meaning (unlike k-‐means)
– Finds variable number of modes – Robust to outliers
• Cons – Output depends on window size – Window size (bandwidth) selecLon is not trivial – ComputaLonally (relaLvely) expensive (~2s/image) – Does not scale well with dimension of feature space
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
3-‐Nov-‐15 49
![Page 50: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/50.jpg)
Lecture 13 - Fei-Fei Li 3-‐Nov-‐15 50
Mean-‐shi4 Algorithm
technical
note
Comaniciu & Meer, 2002
![Page 51: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/51.jpg)
Lecture 13 - Fei-Fei Li 3-‐Nov-‐15 51
Mean-‐shi4 Algorithm
technical
note
Comaniciu & Meer, 2002
![Page 52: Lecture’13:’k,means’and’’ mean,shi4’clustering’vision.stanford.edu/teaching/cs131_fall1516/lectures/... · 2015-11-03 · Fei-Fei Li Lecture 13 - Lecture’13:’k,means’and’’](https://reader034.vdocuments.co/reader034/viewer/2022042310/5ed81f0c0fa3e705ec0de00f/html5/thumbnails/52.jpg)
Lecture 13 - Fei-Fei Li
What will we have learned today
• K-‐means clustering • Mean-‐shi4 clustering
3-‐Nov-‐15 52
Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shi4: A Robust Approach toward Feature Space Analysis, PAMI 2002.