I don't mean to use a tent, just a mat + a sleeping bag, or possibly a bivy sack if the weather is foul.
Yes it is, but only for one night in any single spot. Staying multiple nights is considered camping and is not allowed.
It is never allowed in national parks.
I could only find Dutch references for this, but they all say the same and refer to relevant text in the Austrian lawbooks.
What you describe is absolutely allowed (using a bivouac to spend one night).
If you set up a small camp (described as a "planned bivouac" further in this text), you shouldn't be in a protected area.
Sleeping one night without a tent or a small igloo.
An emergency bivouac is basically allowed everywhere. A planned bivouac is normally tolerated as long as it isn't set up in a protected area (official German term is "Schutzgebiet"). It's explicitly prohibited in such regions (ask the administration of these areas for further information).
Spending one or more nights in a small tent apart from official camping sites.
Protected areas (official German term "Schutzgebiete")
In protected areas camping is generally prohibited. You may ask the administration of the area or of the local commune for an exception. This works surprisingly often.
To check the protected areas of the alps, you can go to www.alparc.org.
In the woods
Camping in the forest is generally prohibited in Germany, Austria, Italy and Switzerland. You've to be permitted by the particular landowner (ask the commune administration).
The detailed regulations differ from state to state in Austria.
You should use a safe camping stove to cook. Open fire is prohibited in protected areas (official German term "Schutzgebiete") and in the forest.
Detailed regulations are very different from region to region.