This article outlines about rounding of numerical results obtained in various analytical methods
1.0 Introduction
In case of analytical methods the results obtained in diluting or weighing are a series of mathematical results with significance. Therefore the digits obtained should be rounded correctly.
1.1 Reported Data
When a number is obtained in calculations the accuracy of the result depends on the accuracy of the number used in calculation. To reduce numerical errors extra significant number is retained in calculations & the final answer is rounded to proper number of significant figures
1.1.1 Considerations
If the extra digit is 5 then increase the previous digit by 1 in case it is odd and don’t change the previous digit
If the extra digit is greater than 5 then leave it and increase the previous digit by 1
If the extra digit is less than 5 then leave the digit
1.2 Monographs as per USP
Numerical results in monograph should include extreme values & all the values in between but not the values outside the limits. This should be applied in rounding of numerical results.
Numerical results are to be reported to the same number of decimal places as limit of expression as stated in monograph.
An explicit statement is made for titrimetric procedures including factors like analyte weight should be measured with precision commensurate and equivalence statement as given in the monograph.
The following table gives USP conventions for rounding, reporting and comparing results with compendial limits.
|
Compendial Requirement |
Unrounded Result |
Rounded result |
Conforms?
|
|
Assay not less than 95.0 and not more than 105.0% of declared |
94.95% |
95.0% |
Y |
|
94.94% |
94.9% |
N |
|
|
105.65% |
105.7% |
N |
|
|
Limit test LTE 0.2% |
0.24% |
0.2% |
Y |
|
0.25% |
0.3% |
N |
1.2.1 Considerations
If the extra digit is greater than 5 then leave it and increase the previous digit by 1
If the extra digit is 5 then drop it and increase the previous digit by 1.
If the extra digit is less than 5 then leave the digit
Retain only one extra digit to right of the rightmost digit of monograph limit expression
1.3 Significant figures
These figures are used to express precision or any uncertainty associated with reported numerical result.
Applying significant figures in spreadsheets, instrumental digital readouts, and handheld calculators is gaining importance due to its capability to produce numbers with precision
1.3.1 Guidelines
All non-zero digits are significant
In a reported result the most significant digit is the left most non zero digit like in 265.871, 2 is the most significant digit
In a reported result if there is a decimal point then the least significant digit is the right most digit (Whether it is Zero or not). E.g. 265.871, 1 is the least significant digit and if there is no decimal point then the right most non zero is the least significant digit.
The number of digits between & including the most & least significant digit is the number of significant digits in result - 265.871 (there are 6 significant digits)
1.4 Significant figures in calculated results
Most of the analytical results are obtained by combinations of arithmetic functions like addition, subtraction, multiplication & division.
1.4.1 Guidelines
i) Addition & Subtraction
In case of addition & subtraction, the answer should have decimal places equal to the component with the least number of decimal places
ii) Multiplication & Division
In case of multiplication and division, the answer should have the same number of significant figures as the number with the fewest significant figures
Rounding of numerical values,analytical methods, significant figures, reported data
