With SHOW PLUGINS, look for rows with a Type value of AUTHENTICATION. Any that have a Library value of NULL are built in and cannot be unloaded.

mysql> SHOW PLUGINS;
+-----------------------+--------+--------------------+---------+---------+
| Name                  | Status | Type               | Library | License |
+-----------------------+--------+--------------------+---------+---------+
| binlog                | ACTIVE | STORAGE ENGINE     | NULL    | GPL     |
| mysql_native_password | ACTIVE | AUTHENTICATION     | NULL    | GPL     |
| mysql_old_password    | ACTIVE | AUTHENTICATION     | NULL    | GPL     |
...

With the INFORMATION_SCHEMA.PLUGINS table, look for rows with a PLUGIN_TYPE value of AUTHENTICATION. Any that have a PLUGIN_LIBRARY value of NULL are built in and cannot be unloaded.

mysql> SELECT * FROM INFORMATION_SCHEMA.PLUGINS
    -> WHERE PLUGIN_TYPE='AUTHENTICATION'\G
*************************** 1. row ***************************
           PLUGIN_NAME: mysql_native_password
        PLUGIN_VERSION: 1.0
         PLUGIN_STATUS: ACTIVE
           PLUGIN_TYPE: AUTHENTICATION
   PLUGIN_TYPE_VERSION: 1.0
        PLUGIN_LIBRARY: NULL
PLUGIN_LIBRARY_VERSION: NULL
         PLUGIN_AUTHOR: R.J.Silk, Sergei Golubchik
    PLUGIN_DESCRIPTION: Native MySQL authentication
        PLUGIN_LICENSE: GPL
           LOAD_OPTION: FORCE
*************************** 2. row ***************************
           PLUGIN_NAME: mysql_old_password
        PLUGIN_VERSION: 1.0
         PLUGIN_STATUS: ACTIVE
           PLUGIN_TYPE: AUTHENTICATION
   PLUGIN_TYPE_VERSION: 1.0
        PLUGIN_LIBRARY: NULL
PLUGIN_LIBRARY_VERSION: NULL
         PLUGIN_AUTHOR: R.J.Silk, Sergei Golubchik
    PLUGIN_DESCRIPTION: Old MySQL-4.0 authentication
        PLUGIN_LICENSE: GPL
           LOAD_OPTION: FORCE