You can delete old export files in Windows Explorer or with the File, Change Export File command - select the file then press the Delete key.
If you delete the active export file, a new empty file (with the same name) will be created automatically the next time you click an Export button.
You can delete any files, including .psz Settings files, in the File dialog, by selecting them then pressing the Delete key.