The trick is to mix it with something else such as,
- Powdered Milk
- Powdered sugar
- Graham Crackers
- Rice Crispies
You can also add other things to increase the sweetness like,
- Chocolate Chips
What you are looking for is for the combination to have the consistency of Play-Doh. What you can do then is wrap the ball in saran wrap to keep it together. This has the advantage of not having it in the plastic container, which doesn't shrink as the peanut butter is eaten.
If you are looking for exact recipes, the thing you are looking for is "No-bake Peanut Butter Balls"