Exploring FE-Toolkit HTML Edge Reports
======================================

| Zeke A. Piskulich\ :sup:`1`, Timothy Giese\ :sup:`1`,  Patricio Barletta\ :sup:`1`, Ryan Snyder\ :sup:`1`, and Darrin M. York\ :sup:`1`
| :sup:`1`\ Laboratory for Biomolecular Simulation Research, Institute
|   for Quantitative Biomedicine and Department of Chemistry and Chemical
|   Biology, Rutgers University, Piscataway, NJ 08854, USA

Learning objectives
-------------------

.. start-learning-objectives

- Learn about and explore HTML edge reports generated with FE-Toolkit.
- Explore comparisons for a real ABFE calculation on Tyk2 ligands

.. end-learning-objectives



Activities
----------
In this Activity, you will learn how to explore HTML edge reports generated with FE-Toolkit. This tutorial will guide you through the process of interpreting the various plots and tables included in the HTML report, and how to use this information to assess the quality of your free energy calculations.

.. start-tutorial

FE-Toolkit HTML Reports
~~~~~~~~~~~~~~~~~~~~~~~

For this tutorial, we have provided the simulation output data from Absolute Binding Free Energy Calculations on Tyk2 for four ligands. In this tutorial, we will primarily use the files for the ligand ejm31; however, we have provided the other ligands as a reference (and as an opportunity for further practice).

The HTML report can be viewed here: `Graph.html <../../../_static/files/ModularTutorials/Alchemical/fe-toolkit/ejm31.html>`_

Header
^^^^^^

.. hint::

   ** Troubleshooting Questions **

   1. Do you have many errors and warnings?
   2. Are you using a recent hash of edgembar?

The first thing you see in an HTML report is the header. It contains important information about the calculation, specifically which version of edgembar was used, what command called edgembar, and what errors and warnings were generated.


.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/header.png
    :alt: Header
    :width: 600px

A few things are worth pointing out: every version of edgembar is given a unique hash (in the above image 3.6.6 H09ccf5), which can be useful for debugging.

Next, a summary of errors is included. These are hyperlinks to the data where the errors occurred. Usually these correspond to less-than-ideal equilibration, although they are not necessarily fully impactful. Errors are shown in red and warnings are shown in yellow.


Objective Function
^^^^^^^^^^^^^^^^^^

The next plot you see in the edge report is the Objective Function that was minimized across all samples to obtain the final value.
This plot is probably the least informative, until you run into a case where your objective function does not look like a parabola.
That's a sign that things went very wrong with the simulation.

.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/objectivefunction.png
    :alt: Header
    :width: 600px


Overall results
^^^^^^^^^^^^^^^

.. hint::

   ** Troubleshooting Questions **

   1. Do your BAR and TI results in the table agree?
   2. How far does your fwdrev analysis deviate from production?
   3. Is there a significant trend in your first and last half analysis?

The next set of data you'll see is a table that includes the values of the free energy (and if you ran edgembar with --fwdrev and --halves, some other analysis too!).


.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/fwdrev.png
    :alt: Header
    :width: 600px

Most important is the table at the bottom, which gives you a summary of the values from analysis with Bennett's Acceptance Ratio (BAR), and Thermodynamic Integration (TI).

.. note::

    A key debugging tool is to check agreement between BAR and TI. If these values do not agree - it suggests a problem with either equilibration, replica exchange acceptances, or alchemical pathway.

With **--fwdrev** and **--halves**, you get the two plots at the top of the image above which include timeseries analysis.

The **fwdrev** analysis compares the free energies calculated from the first X% of the data to those calculated from the last X% of the data.

The **halves** analysis excludes X% of the simulation from start as equilibration, and splits the remaining 100-X% of data into two halves and compares the free energy from each half.

DV/DL Profiles and Replica Exchanges
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. hint::

   ** Troubleshooting Questions **

   1. Is your DV/DL profile smooth? Does it have weird discontinuities?
   2. Are there big differences in your DV/DL profiles between trials?
   3. Does your replica exchange acceptance ratio ever approach a low number (< 0.2?)

The next key set of plots that are found are the DV/DL profile, and the Replica Exchange Acceptance Ratios.


.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/dvdl.png
    :alt: Header
    :width: 600px

In the above image, you can see a well-converged DV/DL profile. Note that the trials track pretty well along each other.

Recall that the dV/dL profile is intrinsically linked to the free energy by the equation:

.. math::

   \Delta G = \int_0^1 \left\langle \frac{\partial V(\mathbf{x}; \lambda)}{\partial \lambda} \right\rangle_{\lambda} d\lambda

.. note::

   For ABFE calculations, the dV/dL profile is shifted by a constant number. This number originates from the correction to the free energy that comes from Boresch Restraints. :doc:`/ModularTutorials/Alchemical/ABFE_of_Tyk2/boresch_contribution_free_energy`.


Now, the replica exchange acceptance ratio is included in the second plot, and it gives you information for each pair of lambda windows what percentage of exchanges were accepted. Here, the value of 0.5 suggests that there were sufficient exchanges.


Data Tables
^^^^^^^^^^^

.. hint::

   ** Troubleshooting Questions **

   1. Did a lot of simulation data get thrown out when selecting production regions?
   2. Did this trial have many (or most) of the warnings?
   3. Do you have single passes and round trips in your calculation?

The next major section of the file is a set of tables (per environment and per trial) that give summary information about the free energy calculation as a function of lambda.

.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/table.png
    :alt: Header
    :width: 600px


There is also a replica exchange data table that provides the data included in the acceptance ratio plot.

.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/repex.png
   :alt: Header
   :width: 600px

The header of this table includes a few additional useful pieces of information:

- **Average single pass num. steps:** The amount of time it takes for a replica to exchange all the way :math:`0 \rightarrow 1` or :math:`1 \rightarrow 0`
- **Round trips/replica:** Total number of round trips divided by the number of replicas.
- **Total num. round trips:** The number of times a replica traverses :math:`0 \rightarrow 1 \rightarrow 0` or :math:`1 \rightarrow 0 \rightarrow 1`

Troubleshooting
^^^^^^^^^^^^^^^

.. hint::

   ** Troubleshooting Questions **

   1. Does the window look converged?
   2. What is the scale of the dependence on the y-axis of the block average

Each lambda window that has a warning also gets a special section which includes information about the convergence of that window.

.. image:: /_static/files/ModularTutorials/Alchemical/fe-toolkit/errors.png
   :alt: Header
   :width: 600px

This shows plots of block averages over time, data tables, etc that can be used for additional debugging.


Wrapping Up
^^^^^^^^^^^

In this activity, you walked through the pieces of an HTML report. Go through the troubleshooting questions at the top of each relevant section with that report. Does this pass as an okay simulation?

.. end-tutorial

Relevant Literature
-------------------

.. footbibliography::