Am I Expecting Too Much

I know that I have very high expectations of my boys with regards to behavior. I know that they are too high, but yet, I can't help myself. I have come to terms with the behavior in church and know they will not be able to just sit and listen until it's time to leave. I don't like it, but yet, so many people in church say they don't mind so both Jay and I have relaxed a bit (not completely, but a bit). I understand they are young children with TONS (and I mean TONS) of energy, so it's hard for them when we go places to not be silly, act up a little and so on, however, I do keep tabs on them and make sure they don't get too out of control. It's hard sometimes because Troy is a follower so if there is someone else doing something, even if he doesn't know them, he'll try it too. And then of course Will just follows whatever big brother is doing and it just gets crazy! But they are just 4 and 23 months after all.
However, I have to draw the line at the dinner table. We've been eating family meals since they were born. We sit at the table, chat, eat and then move on, but lately, this has become miserable. Mainly because Will tries to get a laugh and does something. Then Troy encourages (or lately, instigates and gets Will going) and it gets out of hand. I have tried ignoring Will and getting Troy to ignore him, removing Will from the table for a bit and then trying again, removing him from the table completely and nothing works. We've resorted to our yelling (because we get fed up and finally just snap) which we don't really want to do but I am not lying when I say nothing is working. Finally, today at lunch I had it. They weren't finished eating and Troy was still hungry but they started in with the playing. I gave them both a warning and said if they continued, that was it. No more food, no drink, they were to get away from the table and not allowed to come back and finish or have a snack. Well, right after I said this, guess what happened? Yep, that's right. More bad behavior. So that was it, I did what I said and explained why I was furious. I felt just fine about it too until Troy was whining later about being hungry and wanting to finish his lunch and how he was starving. I didn't give in other than allowing him his milk. He went down for his nap angry at me, refusing to talk to me, not even saying I Love You back to me and turned away when I went to kiss him :( So am I a horrible mommy for expecting good table manners or should I hold strong demanding good behavior even if they go to bed hungry? For those people who always say, "My child knows better than to act up/whine/have bad table manners", how in the world did you get them that way? I certainly feel like we've done a lot and been very strict but it's getting us nowhere????