"Contrast Label Color" not contrasting enough/at all?

[robblob]

I'm on 11.1 R2 and have noticed that the "contrast label color" option for visualizations doesn't really seem to be contrasting the values at all.  I'm setting the properties under the "annotations" section for visualizations, but I'm looking for some insight on how this property is intended to handle the label values.  I've attached an image to show what I'm talking about.