I've just tried that filter out, and it seems that PS goes about this entirely differently. It seems to use a combined RGBCYM-model, since settings for red do not influence magenta or yellow results at all, although red is a part of these in the pure
RGB model. PS is known for compliance with a variety of differing color models (
see this for some of them), and probably uses either Lab or another CIE format internally. So if you want to copy the PS effect, you seem to need to use a different color model all together. Adobe probably documented what they use somewhere...
"When one person suffers from a delusion, it is called insanity. When a million people suffer from a delusion, it is called religion." (Richard Dawkins)