Do the handles show on "Design your field layout" sections?
i.e. When adding fields to entry types in sections?
No, sorry. I agree it'd be useful, though I'm not really sure how to fit into the layout for the Field Layout Designer.
In any case, good feature request – thanks!
@mmikkel Excuse my ignorance at what might be possible with a plugin, but I was thinking something like this?:

<small style="display: block;text-transform: none;font-size: 11px;opacity: 0.5;">headingHandle</small>
Or perhaps just a title tag:

<span title="headingHandle">Heading</span>
Yeah, that could work. I'm not crazy about how much your suggestion would add to the height for for each field; if you have a lot of fields then the scroll-a-mathon of the Field Layout Designer would be even worse. Perhaps a better suggestion would be to make the name clickable; when you click it the name is replaced w/ the handle?
@mmikkel Sounds good. Not too bothered about how it's implemented - always pros and cons, just as long as it's there :)