Use the libjpeg-turbo library
The libjpeg-turbo library creates jpeg in a more efficient way that the default libjpeg library. To use it in GeoServer, two things must be installed: the library itself and the libjpeg-turbo Map Encoder extension
.
Installing the libjpeg-turbo library
To install the library in a Windows machine, follow these steps:
Download the library installer for the libjpeg-turbo download page
Execute the installer and follow the installation instructions.

Installing the Map Encoder extension
Download the extension for the Geoserver version contained in Geoserver Enterprise from this download link
Unzip the downloaded zip file in the
WEB-INF/lib
directory of the GeoServer installation.Restart you GeoServer instance for the changes to take effect.