Depends on how long you sleep for. I collect my 8 hour buildings at 11pm, 7 am and 3pm without issue and normally have 3 collections without problem. Granted you have less of margin for error, but even if you miss the collection its less likely to be robbed than others. Hopefully it'll have a huge xp payout when robbed.