It sounds like you are talking about the practice of dashing off into the woods to sleep in secret wherever you happen to be while traveling. I have heard this called "wild camping" before. Not to be confused with camping on hiking trails where you are allowed to camp anywhere, which I assume you are not talking about because the advice there is generally the opposite of what you have stated.
Most of the reasons that I have seen for making camp late and breaking it early when wild camping in this manner are all variations of "I don't want people knowing I'm there." Often because it is illegal, or because you are leery of crime.
Some questions to ask yourself, to determine your needs and capabilities
Are you able to set up after dark? Either because you can see enough, or because you have light - but then the light source could attract unwanted attention.
How much traffic will the area have? Are you likely to be spotted dashing off into the trees because vehicles are zooming past more than once per minute?
How badly do you want (or need) to stay undiscovered?
And there are more questions you could answer to further develop your personal answer to this.
The answer may also change depending on the site. Does the brush offer such good concealment that you could set up without anyone nearby noticing, or is your activity going to be obviously waving tall grass or bushes about and attracting attention, or possibly your movement be noticeable through the tree line?
When to set up or break camp
So the time should depend on many factors, but there are a few concrete things that we can say:
If there is hardly any traffic nearby and you are very unlikely to be seen by passers, then any time is likely fine.
If it is possible that you might be seen, or if it is of utmost importance to not be discovered (ie: you're in a place where it's illegal, punished harshly, and the law is enforced strongly), then you really have no option but to make/break camp in the dark, and preferably without shining light.
If the situation is somewhere in between the previous two points, as is likely, then waiting until no traffic is in sight to dash off into the trees or bushes to set up quickly, and staying still until dark, is probably sufficient and you would just wait as late as you are willing. And then the opposite for morning: as early as you are willing, which could be after sunrise (stay still after sunrise if so), pack up quickly when no traffic is in sight and dash back onto the road when it feels safe.
Important safety concerns
I have heard of wild campers being discovered and waking up to everything from wild animals nearby, to pleasant landowners wanting to be hospitable sharing coffee and a hot breakfast with the camper, to waking up with a gun in their face held by angry landowners.
Where it's illegal, the obvious expectation is that you'll either be forced to move "anywhere but our town, just get out of here", or arrested; however, I have heard of polite wild campers getting police to leave them alone, and even of police telling the camper where a safer place to camp is and sometimes even checking on the camper in the night to make sure they are safe.
Take this with a grain of salt
I have read up a lot about this practice, and I have been tempted to try it out, but so far I have kept my camping to areas where camping is permitted, such as designated camp sites or in government parks where you can camp anywhere. Your mileage may vary a lot. Be careful!