With advancements in GPU-accelerated free energy methods, it is now possible to obtain sufficiently high precision in free energy calculations to rigorously stress test implementations for consistency, reproducibility and reliability. Herein we  rovide high precision validation tests that examine alchemical transformations of a small molecule data set that has been used elsewhere to examine the reproducibility of free energy calculations across different molecular simulation software  packages. We demonstrate that the most recent, updated AMBER18 provides consistent free energy results in both the gas phase and in solution. We first show, in the context of thermodynamic integration (TI), that results are invariant with  respect to “split” (e.g., stepwise decharge-vdW-recharge) versus “unified” protocols. This brought to light a subtle inconsistency in previous versions of AMBER that was traced to the improper treatment of 1-4 vdW and electrostatic interactions involving atoms across the softcore boundary. We illustrate that, under the assumption that the ensembles produced by different legs of the alchemical transformation between molecules “A” and “B” in the gas phase and aqueous phase are very small, the  inconsistency on the relative hydration free energy is minimal. However, for general cases where the ensembles are shown to be substantially different, these errors can be large. Finally, we demonstrate that results for relative hydration free energy simulations are independent of TI or multistate Bennett’s acceptance ratio (MBAR) analysis, invariant to the specific choice of the softcore region, and agree with results derived from absolute hydration free energy values.