LogAUC - Revision history

Frodo at 17:57, 15 February 2014

2014-02-15T17:57:19Z

← Older revision		Revision as of 17:57, 15 February 2014
Line 28:		Line 28:

	[[Category:Docking]]		[[Category:Docking]]
	~~[[Category:Analysis tools]]~~
	[[Category:Theory]]		[[Category:Theory]]

Frodo at 17:57, 15 February 2014

2014-02-15T17:57:09Z

← Older revision		Revision as of 17:57, 15 February 2014
Line 26:		Line 26:
	==Citation==		==Citation==
	Michael Mysinger, Brian Shoichet. "Rapid Context-Dependent Ligand Desolvation in Molecular Docking". 2010. (in preparation for J Chem Inf Model)		Michael Mysinger, Brian Shoichet. "Rapid Context-Dependent Ligand Desolvation in Molecular Docking". 2010. (in preparation for J Chem Inf Model)

			[[Category:Docking]]
			[[Category:Analysis tools]]
			[[Category:Theory]]

Therese: 5 revisions

2012-10-08T20:29:19Z

5 revisions

← Older revision

Revision as of 20:29, 8 October 2012

Mysinger at 00:37, 13 January 2010

2010-01-13T00:37:18Z

← Older revision		Revision as of 00:37, 13 January 2010
Line 15:		Line 15:
	==Discussion==		==Discussion==

	From similar reasoning based on semilog ROC plots, Clark and Webster-Clark construct the pROC AUC metric<~~sub~~>2</~~sub~~>, which is similar to the numerator of logAUC except that the integration is done over horizontal bars instead of vertical trapezoids. The advantage of constructing logAUC as a fraction over the ideal area is that the choice of base for the logarithm is irrelevant, because changing base simply results in a constant that cancels between numerator and denominator. Also, by explicitly defining the area of interest using λ and integrating vertically, we are able to avoid the singularity at <math>x_i=0</math> encountered in pROC. More importantly, the fixed integration area means we can more directly compare <math>logAUC_\lambda</math> values across databases of different sizes and across targets with different ratios of actives to inactives. The final advantage of logAUC is that if you are used to looking at semilog ROC plots plotted from λ to 1, and understand that logAUC is just the percentage of the total area below the curve, then you can at some point gain the same intuitive feel as AUC has for linear ROC plots. In a semilog ROC plot the random line occupies only a sliver of the total area, and indeed its logAUC is just 14.462%. In order to more easily compare a given logAUC to this random value, we instead report the “adjusted logAUC” as the calculated value minus 14.462%, so that positive values mean overall enrichments better than random.		From similar reasoning based on semilog ROC plots, Clark and Webster-Clark construct the pROC AUC metric<sup>2</sup>, which is similar to the numerator of logAUC except that the integration is done over horizontal bars instead of vertical trapezoids. The advantage of constructing logAUC as a fraction over the ideal area is that the choice of base for the logarithm is irrelevant, because changing base simply results in a constant that cancels between numerator and denominator. Also, by explicitly defining the area of interest using λ and integrating vertically, we are able to avoid the singularity at <math>x_i=0</math> encountered in pROC. More importantly, the fixed integration area means we can more directly compare <math>logAUC_\lambda</math> values across databases of different sizes and across targets with different ratios of actives to inactives. The final advantage of logAUC is that if you are used to looking at semilog ROC plots plotted from λ to 1, and understand that logAUC is just the percentage of the total area below the curve, then you can at some point gain the same intuitive feel as AUC has for linear ROC plots. In a semilog ROC plot the random line occupies only a sliver of the total area, and indeed its logAUC is just 14.462%. In order to more easily compare a given logAUC to this random value, we instead report the “adjusted logAUC” as the calculated value minus 14.462%, so that positive values mean overall enrichments better than random.

	<math>Adjusted~LogAUC=LogAUC_{0.001}-.14462</math>		<math>Adjusted~LogAUC=LogAUC_{0.001}-0.14462</math>

	==References==		==References==
Line 23:		Line 23:
	## Clark, R. D.; Webster-Clark, D. J., Managing bias in ROC curves. J Comput Aided Mol Des 2008, 22, (3-4), 141-6.		## Clark, R. D.; Webster-Clark, D. J., Managing bias in ROC curves. J Comput Aided Mol Des 2008, 22, (3-4), 141-6.
	## Truchon, J. F.; Bayly, C. I., Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem. J Chem Inf Model 2007, 47, (2), 488-508.		## Truchon, J. F.; Bayly, C. I., Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem. J Chem Inf Model 2007, 47, (2), 488-508.

			==Citation==
			Michael Mysinger, Brian Shoichet. "Rapid Context-Dependent Ligand Desolvation in Molecular Docking". 2010. (in preparation for J Chem Inf Model)

Mysinger at 00:32, 13 January 2010

2010-01-13T00:32:03Z

← Older revision		Revision as of 00:32, 13 January 2010
Line 1:		Line 1:
	==What is LogAUC?==		==What is LogAUC?==

	LogAUC is a metric to evaluate virtual screening performance that has ~~some nice characteristics. It~~ is ~~intuitive~~ to ~~use~~		LogAUC is a metric to evaluate virtual screening performance that has many of the same advantages as area under the curve (AUC), but is based on a plot where the x-axis is semilog in order to focus on early enrichment.

	==Motivation==		==Motivation==
Line 9:		Line 9:
	==Definition==		==Definition==

	Formally, we define <math>logAUC_\lambda</math>, where the log area computations run from <math>\lambda</math> to 1.0, and we typically refer to <math>logAUC_{0.001}</math> as simply logAUC, where the area is integrated from 0.1 percent (0.001) to 100 percent (1.0) of decoys found. For integrating the area under the curve, we use the trapezoidal rule as follows:		Formally, we define <math>logAUC_\lambda</math>, where the log area computations run from <math>\lambda</math> to 1.0, and we typically refer to <math>logAUC_{0.001}</math> as simply <math>logAUC</math>, where the area is integrated from 0.1 percent (0.001) to 100 percent (1.0) of decoys found. For integrating the area under the curve, we use the trapezoidal rule as follows:

	<math>LogAUC_\lambda=\frac{\displaystyle \sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}</math>		<math>LogAUC_\lambda=\frac{\displaystyle \sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}</math>

			==Discussion==

			From similar reasoning based on semilog ROC plots, Clark and Webster-Clark construct the pROC AUC metric<sub>2</sub>, which is similar to the numerator of logAUC except that the integration is done over horizontal bars instead of vertical trapezoids. The advantage of constructing logAUC as a fraction over the ideal area is that the choice of base for the logarithm is irrelevant, because changing base simply results in a constant that cancels between numerator and denominator. Also, by explicitly defining the area of interest using λ and integrating vertically, we are able to avoid the singularity at <math>x_i=0</math> encountered in pROC. More importantly, the fixed integration area means we can more directly compare <math>logAUC_\lambda</math> values across databases of different sizes and across targets with different ratios of actives to inactives. The final advantage of logAUC is that if you are used to looking at semilog ROC plots plotted from λ to 1, and understand that logAUC is just the percentage of the total area below the curve, then you can at some point gain the same intuitive feel as AUC has for linear ROC plots. In a semilog ROC plot the random line occupies only a sliver of the total area, and indeed its logAUC is just 14.462%. In order to more easily compare a given logAUC to this random value, we instead report the “adjusted logAUC” as the calculated value minus 14.462%, so that positive values mean overall enrichments better than random.

			<math>Adjusted~LogAUC=LogAUC_{0.001}-.14462</math>

	==References==		==References==
	1. Nicholls, A., What do we know and when do we know it? J Comput Aided Mol Des 2008, 22, (3-4), 239-55.		## Nicholls, A., What do we know and when do we know it? J Comput Aided Mol Des 2008, 22, (3-4), 239-55.
	2.		## Clark, R. D.; Webster-Clark, D. J., Managing bias in ROC curves. J Comput Aided Mol Des 2008, 22, (3-4), 141-6.
			## Truchon, J. F.; Bayly, C. I., Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem. J Chem Inf Model 2007, 47, (2), 488-508.

Mysinger at 00:15, 13 January 2010

2010-01-13T00:15:46Z

← Older revision		Revision as of 00:15, 13 January 2010
Line 5:		Line 5:
	==Motivation==		==Motivation==

	When we look at virtual screening performance, we plot an ROC curve (or enrichment curve) with a base 10 semilog x-axis, because this has the advantage of focusing the graph on "early enrichment", where molecules are most likely to be selected for further testing. If we had instead plotted the curve with the usual linear x-axis, then the area under the curve (AUC) is a well-regarded metric to summarize the overall performance of a virtual screening campaign as a single number<sup>1</sup>. While ~~ROC~~ AUC can be formulated alternate ways, it can be		When we look at virtual screening performance, we plot an ROC curve (or enrichment curve) with a base 10 semilog x-axis, because this has the advantage of focusing the graph on "early enrichment", where molecules are most likely to be selected for further testing. If we had instead plotted the curve with the usual linear x-axis, then the area under the curve (AUC) is a well-regarded metric to summarize the overall performance of a virtual screening campaign as a single number<sup>1</sup>. While AUC can be formulated alternate ways<sup>2,3</sup>, it can be mechanically constructed by simply integrating under the curve, and interpreted as the fraction of the area under the curve over the area under the best possible ROC curve. It just happens that in a linear ROC plot, the AUC of the best possible curve is the entire unit square, with an area of 1. By analogy, in our typical semilog plots, we can construct the same fraction of the area under the log curve, over the area under the perfect log curve, and define that fraction as the logAUC. The lone nuisance is that the area under the log curve is infinite in general. However, if we are practical and limit our focus to a region of log space that we can actually measure, say above a certain threshold <math>\lambda</math>, then the perfect log area is finite.

			==Definition==

			Formally, we define <math>logAUC_\lambda</math>, where the log area computations run from <math>\lambda</math> to 1.0, and we typically refer to <math>logAUC_{0.001}</math> as simply logAUC, where the area is integrated from 0.1 percent (0.001) to 100 percent (1.0) of decoys found. For integrating the area under the curve, we use the trapezoidal rule as follows:

	<math>LogAUC_\lambda=\frac{\displaystyle \sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}</math>		<math>LogAUC_\lambda=\frac{\displaystyle \sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}</math>

	==References==		==References==
			1. Nicholls, A., What do we know and when do we know it? J Comput Aided Mol Des 2008, 22, (3-4), 239-55.
			2.

Mysinger at 00:00, 13 January 2010

2010-01-13T00:00:40Z

← Older revision		Revision as of 00:00, 13 January 2010
Line 1:		Line 1:

	==What is LogAUC?==		==What is LogAUC?==

Line 6:		Line 5:
	==Motivation==		==Motivation==

	When we look at virtual screening performance, we plot an ROC curve (or enrichment curve) with a base 10 semilog x-axis, because this has the advantage of focusing the graph on "early enrichment", where molecules are most likely to be selected for further testing. If we had instead plotted the curve with the usual linear x-axis, then the area under the curve (AUC) is a well-regarded metric to summarize the overall performance of a virtual screening campaign as a single number.		When we look at virtual screening performance, we plot an ROC curve (or enrichment curve) with a base 10 semilog x-axis, because this has the advantage of focusing the graph on "early enrichment", where molecules are most likely to be selected for further testing. If we had instead plotted the curve with the usual linear x-axis, then the area under the curve (AUC) is a well-regarded metric to summarize the overall performance of a virtual screening campaign as a single number<sup>1</sup>. While ROC AUC can be formulated alternate ways, it can be

			<math>LogAUC_\lambda=\frac{\displaystyle \sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}</math>

			==References==

Mysinger at 23:41, 12 January 2010

2010-01-12T23:41:33Z

New page

==What is LogAUC?==

LogAUC is a metric to evaluate virtual screening performance that has some nice characteristics. It is intuitive to use

==Motivation==

When we look at virtual screening performance, we plot an ROC curve (or enrichment curve) with a base 10 semilog x-axis, because this has the advantage of focusing the graph on "early enrichment", where molecules are most likely to be selected for further testing. If we had instead plotted the curve with the usual linear x-axis, then the area under the curve (AUC) is a well-regarded metric to summarize the overall performance of a virtual screening campaign as a single number.