No it doesn't sound silly, actually it's a good idea in my opinion.
I think you'll have to buy the door handle cover.
I just can't find the proper part number, because I'm noob. I think one of these:
82652G3030 L
82652G4040 L
82662G3020 R
82662G4010 R
It's complicated because the L and R depends on country, I mean Ozzyland and UK have the opposite driver side. If you order the EU part number to your driver side (L) that won't be holed because in EU the driver sits on the R side.
You have to order at the dealer/garage anyway, they will help with part# I think. And it will arrive as unpainted plastic, so you'll have to order a painting job to your car's body color.