Some Android models come with a Ambient light sensor that adjusts the back light depending on the ambient light (duh :p) while a nice feature it does however it doesn't play well with the setBrightness function.
Currently the only way to override this behaviour seems to be to check Disable auto back light under Sound & Display in settings.
Confirmed to work HTC Hero, once my Nexus One arrives I'll see if it has the same behaviour. Googling tells me that it might also work for Droid.
Update 2010-03-08:
Revived my Nexus One, it comes with automatic brightness disabled, however setBrightness still works as intended with automatic brightness enabled.