How do I do the emergency turn off Android device with embedded battery?

Imagine your device is hacked and you want to turn it off now (because of it is using network in bad manner), but the rootkit blocked all functions.

How do I forcibly turn off the device? I've tried holding the power button and the devide said it will reboot instead of shutting down.