Current DCGM agent seems to only open a TCP socket and let controller use dcgmconnectv2 to connect. This makes any user who knows the DCGM agent service IP and port be able to connect to and do anything libdcgm provides.
To avoid this, I think it may be possible to do something to the following code which handles login for authentication.
https://github.com/NVIDIA/DCGM/blob/master/modules/core/DcgmModuleCore.cpp#L1652