I would recommend to set the transaction logging to: "Check file size when extending the file" but keep in mind that you must then enable also a program document which run compact with the -B in the night. If you wish to enable DAOS with 8.5 please check technote #1381303.
If you set the transaction logging to "Check file size when extending the file" the following will happen during mail routing to the users on that server:
The Router calculates the current size of a mail file from its actual physical size. The calculated physical size includes the unused "white space" in a file that results when a user deletes or archives a message. Domino does not immediately recover this white space. As a result, accumulated white space may account for a large portion of the file size, so that the actual mail file size is considerably larger than the combined size of its stored messages.
The size check occurs only if adding a message requires an increase in the size of the mail file. When quota enforcement is enabled and this option is selected, if a message delivered to the mail file requires an increase in the file size that would result in a quota violation, delivery fails. However, a message is always delivered if there is sufficient white space to accommodate it.