You may be doing this anyway, but as soon as you feel a take/rattle, let the plenty of slack line out to give the flattie plenty of time to take the bait.
By plenty I mean several metres off the reel. Sometimes it’s worth doing a couple of times to ensure the fish has lots of time.
Obviously this applies more to drifting, but even at anchor I’d leave it a few minutes.
When drifting, most flatties will attack a moving bait, but if you keep the line tight it will just pull the bait away and past the fish. They may follow and keep nibbling, but you’re unlikely to get a hook up that way, they need time to get it down them.
Never strike a flattie bite - when you think/hope it’s hooked, just ease into the fish gently and start winding.
I don’t know your patch but the other aspect of course is you may not be fishing a productive area for flatties and the bites are something else.
If you’re not hooking anything on the drift bites though, I think yanking the bait away from the fish is the issue.
