As Jim Rhoades noted in one of the comments, you can declare support for custom font sizes, but leave out the array of font size options. This removes the "Typography" section entirely so there is no way to choose a font size. The code looks like this:
add_theme_support( 'editor-font-sizes' );For those looking for all the code needed to make it work, you would call this from the after_setup_theme hook inside the functions.php file within your theme:
function theme_setup() { add_theme_support( 'editor-font-sizes' );}add_action( 'after_setup_theme', 'theme_setup' );I've tested this in WordPress 5.7, but it's possible it's supported before this version.