Diagnostic accuracy of the gravity stress test and clinical signs in cases of isolated supination–external rotation-type lateral malleolar fractures

(1)

Diagnostic accuracy of the gravity stress test and clinical signs in cases of isolated

supination–external rotation-type lateral malleolar fractures

S. Nortunen, T. Flinkkilä, I. Lantto, T. Kortekangas, J. Niinimäki, P. Ohtonen, H. Pakarinen From Oulu

University Hospital, Oulu, Finland

S. Nortunen, MD, Orthopaedic surgeon, Department of Surgery

T. Flinkkilä, MD, PhD, Orthopaedic Surgeon, Department of Surgery

I. Lantto, MD, Orthopaedic Surgeon, Department of Surgery

T. Kortekangas, MD, Orthopaedic Surgeon, Department of Surgery

J. Niinimäki, MD, PhD, Radiologist, Department of Diagnostic Radiology

P. Ohtonen, McS, Biostatistician, Departments of Anesthesiology and Surgery

H. Pakarinen, MD, PhD, Orthopaedic Surgeon, Head of Trauma, Department of Surgery Oulu University Hospital, PL 21, FI 90029 OYS, Oulu, Finland.

Correspondence should be sent to Dr S. Nortunen; e-mail:

simo.nortunen@ppshp.fi

doi:10.1302/0301-620X.97B8.

35062 $2.00 Bone Joint J 2015;97-B:1126–31.

Received 2 September 2014;

Accepted after revision 5 March 2015

We prospectively assessed the diagnostic accuracy of the gravity stress test and clinical findings to evaluate the stability of the ankle mortise in patients with supination–external rotation-type fractures of the lateral malleolus without widening of the medial clear space.

The cohort included 79 patients with a mean age of 44 years (16 to 82). Two surgeons assessed medial tenderness, swelling and ecchymosis and performed the external rotation (ER) stress test (a reference standard). A diagnostic radiographer performed the gravity stress test.

For the gravity stress test, the positive likelihood ratio (LR) was 5.80 with a 95%

confidence interval (CI) of 2.75 to 12.27, and the negative LR was 0.15 (95% CI 0.07 to 0.35), suggesting a moderate change from the pre-test probability. Medial tenderness, both alone and in combination with swelling and/or ecchymosis, indicated a small change (positive LR, 2.74 to 3.25; negative LR, 0.38 to 0.47), whereas swelling and ecchymosis indicated only minimal changes (positive LR, 1.41 to 1.65; negative LR, 0.38 to 0.47).

In conclusion, when gravity stress test results are in agreement with clinical findings, the result is likely to predict stability of the ankle mortise with an accuracy equivalent to ER stress test results. When clinical examination suggests a medial-side injury, however, the gravity stress test may give a false negative result.

Cite this article: Bone Joint J 2015; 97-B:1126–31.

In cases of isolated Lauge–Hansen¹ supination–

external rotation (SER)-type fractures of the lateral malleolus, the assessment of ankle stability is challenging. Such fractures can be either stable or unstable, depending on the state of the deltoid ligament, which controls the external rotation of the talus.^2-6 In the absence of radiographic evidence of medial widening of the ankle mortise, clinical signs of medial-side injury alone are insufficient,^7-9 and so other methods are needed to guide the decision for operative or non-operative treatment.

The external rotation (ER) stress test is considered the best method for assessing ankle mortise stability^5-14 and is commonly used in clinical practice. It has several problems, however: it must be performed by an experienced examiner; the force applied can vary between surgeons; it causes pain and discomfort to the patient;¹² and the examiner is exposed to radiation. Other described examination techniques include MRI^13,15 and arthroscopy,¹⁶ but these methods are either too costly (MRI) or too invasive (arthroscopy) for routine use.

It has been suggested that the gravity stress test may demonstrate an unstable ankle fracture as reliably as the ER stress test.^10,12,17 This

has several advantages: the surgeon is not exposed to radiation, there may be less oppor- tunity for human error in interpretation, it sub- jects the patient to less pain,¹² and it may enable decision making in emergency departments without surgical consultation. There is however insufficient evidence currently to sup- port its use.

In this prospective study, we evaluated the accuracy of clinical signs and the gravity stress test in comparsion with the ER stress test for diagnosing instability of the ankle mortise in SER-type unimalleolar ankle fractures without overt medial widening on initial radiographs.

Our hypothesis was that the ER stress test could be replaced by the gravity stress test in clinical practice.

Patients and Methods

The study included skeletally mature patients (≥ 16 years old) with a unilateral Lauge–

Hansen¹ SER-type ankle fracture with no medial widening or incongruity on standard ankle radiographs, who were treated within a week of injury at our hospital between March 2012 and April 2013. Patients were excluded if they had bilateral ankle fractures, pathological

(2)

fractures, concomitant tibial shaft fractures, previous significant injury to or fracture of either ankle, peripheral neuropathy, soft-tissue infection in the region of the injured ankle, or if they were unable to walk unaided before injury.

Fractures were classified according to the Lauge–Hansen classification by the senior orthopaedic trauma surgeon responsible for each patient’s care.

A total of 96 patients were screened, eight of whom were excluded because of a previous ankle injury, two for inability to complete the study protocol, one for inability to walk unaided before injury and one for peripheral neuropathy. Two patients declined to participate and three had unsatisfactory gravity radiographs owing to failure to obtain a mortise view, thereby preventing accurate measurement of radiological parameters. The final study group comprised 79 consecutive patients with a mean age of 44 years (16 to 82), 37 of whom were women whose mean age was 50 years (16 to 82) and 42 were men with a mean age of 39 years (16 to 79) (Fig. 1).¹⁸

The study protocol was approved by the local ethics review board, and each patient gave informed consent before study participation.

Clinical examination. Before the stress tests the patients were examined by two surgeons for swelling, tenderness and ecchymosis around the medial malleolus. Clinical findings were recorded as positive or negative.

Stress tests. For all patients, manual ER stress tests^10,12 were performed by two senior orthopaedic trauma surgeons, or by one senior orthopaedic trauma surgeon and one senior orthopaedic trauma resident who had completed our trauma training (Fig. 2). A total of ten surgeons were involved with ER stress testing. Prior to testing, the princi- ples of the manual ER stress test were summarised and rehearsed. The tests were performed independently by two surgeons, one performing the test and the other evaluating the resulting digital radiograph. Each result was recorded as positive (medial clear space (MCS) ≥ 5 mm) or negative (MCS < 5 mm) and the surgeons were blinded to each other’s assessment. For the gravity stress test, a diagnostic radiographer performed imaging according to the protocol described by Michelson, Varner and Checcone¹⁷ (Fig. 3).

Radiographic measurements. The measurements for all three images were adjusted for magnification, using a con- stant 115 cm source-to-detector distance and a standard 30 mm radiographic marker. To measure the MCS, one author (SN), who was blinded to the clinical findings, analysed all radiographs twice using a diagnostic workstation and a high-resolution monitor. Measurements were recorded to the nearest millimetre. The examiner’s assessment and initial measurement were used to calculate the interobserver reliability of the ER stress test. Two authors (SN, TK), who were blinded to each other’s assessment and to the clinical Patients refused to

participate (n = 2)

Excluded patients (n = 12) - Previous ankle injury (n = 8)

- Inability to comply with protocol (n = 2) - Inability to walk without aid prior to fracture (n = 1)

- Peripheral neuropathy (n = 1) Eligible patients

(n = 96)

Gravity stress test (n = 82)

Negative (n = 39)

Inconclusive (not mortise) (n = 3)

Positive (n = 40 )

No ER stress test (n = 0)

No ER stress test (n = 0) No ER stress test

(n = 0)

Inconclusive (n = 0)

Inconclusive (n = 0) Inconclusive

(n = 0)

ER+

(n = 5) ER+

(n = 34)

ER−

(n = 6)

ER−

(n = 34)

ER+

(n = 2)

ER−

(n = 1) ER stress test (reference)

(n = 39)

ER stress test (reference) (n = 3)

ER stress test (reference) (n = 40)

Fig. 1

Standards for Reporting of Diagnostic Accuracy flow diagram.¹⁸ ER, external rotation.

(3)

findings, performed measurements of the gravity stress test radiographs to calculate the interobserver reliability. Disa- greements were resolved by a consensus decision. The MCS was measured as the distance between the lateral border of the medial malleolus and the medial border of the talus at the level of the talar dome.¹⁹ The tests were considered positive (indicating ankle instability) if the MCS was ≥ 5 mm and at least 1 mm greater than the superior tibiotalar clear space.^7,20 When MCS values differed between the two ER stress test radiographs, the highest value was used.

Statistical methods. Measurements are expressed as the mean and range. Sensitivity, specificity and positive and negative likelihood ratios (LR) with 95% confidence inter-

vals (CI) were calculated for clinical examination and the gravity stress test, using the manual ER stress test as a reference. In theory, an SER fracture can be either stable or unstable, with a probability of instability of 0.5. The probability is recalculated after each test, as new clinical or radiological evidence may increase (LR > 1) or reduce (LR < 1) the probability. Knowing the incidence of ER stress-positive injuries in our study group, we used it as the pre-test probability (before clinical examination or any additional imaging). Positive and negative LRs were used to calculate post- test probabilities (after clinical examination and imaging) for clinical findings, gravity stress testing and combinations thereof. The LRs were interpreted as described by Jaeschke, Guyatt and Sackett.²¹ The relevance of LRs to post-test probability is considered very small for LRs of 0.5 to 2, small for LRs of 2 to 5 and 0.2 to 0.5, and moderate for LRs of 5 to 10 and 0.1 to 0.2, whereas an LR > 10 or < 0.1 can often conclusively confirm or exclude the diagnosis.²¹

Reliability between the gravity and ER stress tests was calculated based on kappa (k) coefficients. The interobserver reliability of the ER stress test was assessed using k coefficients and the percentage agreement, enabling an overview of the agreement between two assessors without accounting for any agreement that occurs purely by chance.

The k coefficient was interpreted following the guidelines of Landis and Koch,²² where k < 0.20 represents slight reliability, 0.21 to 0.40 fair reliability, 0.41 to 0.60 moderate reliability, 0.61 to 0.80 substantial reliability and > 0.80 almost perfect reliability.

One-way analysis of variance (ANOVA) was used to calculate associations between clinical signs, stress tests and the delay between injury and clinical evaluation, as well as between patient age and clinical signs. A p-value < 0.05 was considered statistically significant.

Results

The mean delay from injury to clinical evaluation and stress tests was 2.3 days (0 to 6). A total of 29 patients (41%) were evaluated and tested within 24 hours after injury. The gravity stress test had a sensitivity of 0.87 and specificity of 0.85 for diagnosing instability of the ankle mortise. The positive LR of the gravity stress test was 5.80 (95% CI 2.75 to 12.27) and the negative LR was 0.15 (95% CI 0.07 to 0.35). ER stress test results were positive in 39 cases (49%) and gravity stress test results were positive in 40 cases (51%) (Table I).

For medial tenderness and its combinations with swelling and ecchymosis, positive LR values ranged from 2.74 to 3.25 and negative LR values from 0.50 to 0.62. For swelling and ecchymosis around the medial malleolus and their combination, positive LR values varied from 1.41 to 1.65 and negative LR values from 0.38 to 0.47 (Table II).

The k coefficient for gravity and ER stress tests was 0.72, indicating substantial reliability. Interobserver reliability was 91% for the ER stress test and 95% for the gravity stress test, and k coefficients were 0.82 (substantial reliability) and 0.90 (almost perfect reliability), respectively.

Fig. 2

Photograph of the manual external rotation stress test. The tibia was stabilised with one hand and internally rotated 10° to 15° to provide a true mortise view. The ankle was then positioned in neutral flexion and external rotation force was applied to the forefoot with the other hand.^10,12

Fig. 3

Photograph of the gravity stress test. Patients lay on the injured side with a mount placed under the leg, which was internally rotated 10° to 15° and avoiding plantar flexion of the foot, a mortise view in the hori- zontal radiograph was acquired.

(4)

The incidence of ankle mortise instability in our study population was 0.49. The probability of an ER stress test- positive fracture was 0.85 after a positive gravity stress test and 0.13 after a negative test. If medial tenderness was pre- sent, this probability increased to 0.94 to 0.95 after a positive gravity stress test and decreased to 0.29 to 0.32 after a negative gravity stress test. Without clinical findings, the probability of an ER stress test-positive fracture was 0.77 after a positive gravity stress test and 0.08 after a negative gravity stress test (Table II).

No significant associations were found between the ER stress test results and the gravity stress test results (p = 0.11), medial tenderness (p = 0.83), or delay between injury and evaluation (p = 0.21). Delay between injury and evaluation was associated with swelling (p = 0.01) and ecchymosis (p = 0.004). Patient age was not associated with medial tenderness (p = 0.58) or swelling (p = 0.32), but was associated with ecchymosis (p = 0.005).

Adverse effects. One patient with an MCS of 13 mm in both ER and gravity tests exhibited static talar shift and subluxation of the joint after the tests, which was not evi- dent before testing.

Discussion

The gravity stress test showed good sensitivity and specificity for diagnosing unstable unimalleolar SER-type fractures of the ankle mortise. A positive LR increased the probability of an unstable (ER stress-positive) injury only moderately, from 0.49 to 0.85, whereas a negative LR reduced the probability moderately to 0.13 after gravity stress testing.

The gravity stress test showed good reliability with the

manual ER stress test used as the reference. Both the gravity and the ER stress tests showed excellent interobserver reliability. Clinical findings on the medial side of the ankle alone had insufficient diagnostic value for the assessment of ankle mortise stability. However, when clinical signs (especially medial tenderness) were in agreement with the gravity stress test result, there was an excellent probability that the result would be equivalent to the ER stress test result. If clinical examination and the gravity stress test results are not in agreement, an ER stress test should be considered. Our results did not confirm our hypothesis that the gravity test could completely replace the ER stress test.

Michelson et al¹⁷ first described the gravity stress test in eight cadaver ankles, demonstrating that this test could identify ankle mortise instability (lateral shift ≥ 2 mm and a valgus tilt ≥ 15°) when the deep deltoid ligament was dis- sected. Subsequently, Gill et al¹⁰ and Schock et al¹² reported statistically significant correlations between the ER and gravity stress tests. However, the correlation coefficient measures the strength of the relationship between two tests, not the agreement between them. Plotting the MCS results of both gravity and ER stress tests illustrates the fundamental difference between correlation and agreement. Any straight line would indicate perfect correlation, whereas agreement would only be perfect if the values of both measurements were identical. Therefore, presentation of the correlation between two diagnostic tests might be misleading and is not recommended.²³ The authors did not report the sensitivity of the gravity stress test, nor its specificity, therefore we could not calculate the LRs.

Table I. Cross-tabulation of medial clear space in external rotation (ER) and gravity stress tests

Gravity stress test

Negative (< 5 mm) Positive (≥ 5 mm)

ER stress test Negative (< 5 mm) 34 6

Positive (≥ 5 mm) 5 34

Table II. Sensitivity, specificity, positive likelihood ratio (LR+) and negative (LR−) (with 95% CI), and post-test probability values for clinical signs and gravity stress test for an external rotation stress test–positive injury (pretest probability 0.49)

Tests^*

Sens-itivity Spec-ificity LR+ (95% CI) LR− (95% CI)

Post-test probability

Gravity+ Gravity−

Gravity 0.87 0.85 5.81 (2.75 to 12.27) 0.15 (0.07 to 0.35) 0.85 0.13

Clinical tests Clinical+ Clinical− Clinical+ Clinical−

Swelling (n = 57) 0.85 0.4 1.41 (1.06 to 1.9) 0.38 (0.17 to 0.88) 0.89 0.69 0.21 0.06

Medial tenderness (n = 33) 0.62 0.78 2.74 (1.46 to 5.12) 0.5 (0.32 to 0.76) 0.94 0.74 0.29 0.07

Ecchymosis (n = 49) 0.77 0.53 1.62 (1.12 to 2.34) 0.44 (0.23 to 0.84) 0.9 0.71 0.19 0.06

Swelling and tenderness

(n = 29) 0.56 0.83 3.22 (1.56 to 6.67) 0.53 (0.36 to 0.78) 0.95 0.75 0.32 0.07

Swelling and ecchymosis

(n = 47) 0.74 0.55 1.65 (1.12 to 2.44) 0.47 (0.26 to 0.85) 0.9 0.73 0.19 0.06

Tenderness and ecchymosis

(n = 26) 0.49 0.83 2.78 (1.32 to 5.87) 0.62 (0.44 to 0.87) 0.94 0.78 0.29 0.08

All clinical signs (n = 25) 0.49 0.85 3.25 (1.45 to 7.26) 0.6 (0.43 to 0.84) 0.95 0.77 0.32 0.08

* n, number of positive finding CI, confidence interval

(5)

Gill et al¹⁰ screened 60 patients who met their inclusion criteria, but included only 25 (13 stable and 12 unstable ankles) in their study, demonstrating a possible selection bias. Their criteria for the diagnosis of an unstable ankle mortise were an MCS ≥ 4 mm and at least 1 mm greater than the superior tibiotalar clear space. Schock et al¹² used similar values in a series of 29 patients (13 stable and 16 unstable).

Egol et al⁷ examined 101 patients with isolated SER-type fractures of the lateral malleolus without MCS widening and found that 65% were unstable (MCS ≥ 4 mm). They concluded that, if evaluated within 24 hours of injury, the clinical signs were insufficiently sensitive to predict instability.

The LRs calculated on the basis of their results are similar to ours. Their positive LR for the combination of tenderness and ecchymosis was 6.67, which offered greater accuracy than our positive LR for the gravity stress test. DeAngelis et al⁸ evaluated 55 patients with isolated SER-type fracture of the lateral malleolus without widening, and ER stress tests revealed that 42% were unstable (MCS > 4 mm). They found no statistically significant association between tenderness and ankle mortise stability. Positive and negative LR values calculated based on their data were 1.39 and 0.73, respectively. McConnell et al⁹ also concluded that clinical signs were not accurate predictors of instability.

LRs could not be calculated from their results.

Swelling and ecchymosis appear during the first days after injury, supporting the association we have described between clinical findings and delayed examination. Older patients demonstrated ecchymosis on the medial side of the ankle more commonly, probably owing to thinner and more fragile soft-tissues. Medial tenderness was independ- ent of length of delay and patient age, and the positive LR of tenderness alone or in combination with other clinical signs was more accurate than with swelling and ecchymosis. Owing to the low positive LRs, we agree with the con- clusions of previous studies^7-9 that medial signs alone during clinical examination are insufficient predictors of instability. However, in combination with a gravity stress test, they become important in the assessment of stability.

When clinical findings contradict gravity stress test results, the ER stress test should be considered.

The strengths of our study include its prospective design, reasonably large patient cohort, blinded ER and gravity stress tests, and blinded analysis of stress tests with respect to the clinical findings. The LRs describe the clinical value of a diagnostic test better than positive or negative predic- tive values, since the incidence of unstable versus stable ankles does not affect them but does impact predictive values.²⁴ This study also has several limitations. ER stress tests were conducted before gravity testing to avoid performance bias during ER testing and to enable blinded assessment of the resulting radiographs. The displacing force during ER stress testing may have influenced the gravity test results.

Additionally, surgeons assessed the clinical signs together and recorded only a consensus decision. The gravity stress

test was performed only once per patient; however, the quality of the radiograph was ensured, and it is unlikely that repeated gravity testing would change the result if a true mortise view is achieved. ER stress radiographs were analysed by a single investigator but the measurements were performed twice on diagnostic workstations with high-definition grey-scale displays, and it is unlikely that the interobserver reliability of these radiographic measurements would differ from that of the gravity stress test. The wide CIs of the LRs for the gravity stress test and clinical signs indicate that the results must be evaluated cautiously.

The ER stress test is the most discussed method in the literature for evaluating ankle mortise stability in patients with a SER-type injury,^5-14 but its clinical value remains contro- versial. Non-operative and operative treatment of ER stress- positive injuries may result in similar functional outcomes at one year post-operatively.²⁰ Both ER and gravity stress tests may exaggerate the need for operative treatment when com- pared with weight-bearing radiographs.^25,26 Further investi- gation is needed to evaluate which of these stress tests is truly prognostic and best adopted by clinicians.

In conclusion, when gravity stress test results are in agreement with clinical findings, the result is likely to reveal the stability of the ankle mortise with accuracy equivalent to ER stress test results. When clinical examination suggests injury to the medial side, however, a gravity stress test may give false negative result.

Supplementary material

A section explaining definitions of terms and equa- tions used with regard to likelihood ratios in diagnostic tests, and a table of likelihood ratios and their interpretation, is available alongside the online version of this article at www.bjj.boneandjoint.org.uk

Author contributions:

S. Nortunen: Design, Data collection, Data analysis, Writing the paper.

T. Flinkkilä: Design, Data analysis, Writing the paper.

I. Lantto: Design, Data collection, Writing the paper.

T. Kortekangas: Data collection, Writing the paper.

J. Niinimäki: Design, Data collection, Writing the paper.

P. Ohtonen: Design, Data analysis, Writing the paper H. Pakarinen: Design, Data collection, Writing the paper.

No benefits in any form have been received or will be received from a commer- cial party related directly or indirectly to the subject of this article.

This article was primary edited by P. Page and first proof edited by G. Scott.

References

1. Lauge-Hansen N. Fractures of the ankle. II. Combined experimental-surgical and experimental-roentgenologic investigations. Arch Surg 1950;60:957–985.

2. Harper MC. An anatomic study of the short oblique fracture of the distal fibula and ankle stability. Foot Ankle 1983;4:23–29.

3. Rasmussen O, Kromann-Andersen C, Boe S. Deltoid ligament. Functional analy- sis of the medial collateral ligamentous apparatus of the ankle joint. Acta Orthop Scand 1983;54:36–44.

4. Michelson JD. Fractures about the ankle. J Bone Joint Surg [Am] 1995;77-A:142–

152.

5. Michelson JD, Magid D, McHale K. Clinical utility of a stability-based ankle frac- ture classification system. J Orthop Trauma 2007;21:307–315.

(6)

6. Pakarinen H, Flinkkilä T, Ohtonen P, et al. Intraoperative assessment of the sta- bility of the distal tibiofibular joint in supination-external rotation injuries of the ankle:

sensitivity, specificity, and reliability of two clinical tests. J Bone Joint Surg [Am]

2011;93-A:2057–2061.

7. Egol KA, Amirtharajah M, Tejwani NC, Capla EL, Koval KJ. Ankle stress test for predicting the need for surgical fixation of isolated fibular fractures. J Bone Joint Surg [Am] 2004;86-A:2393–2398.

8. DeAngelis NA, Eskander MS, French BG. Does medial tenderness predict deep deltoid ligament incompetence in supination-external rotation type ankle fractures? J Orthop Trauma 2007;21:244–247.

9. McConnell T, Creevy W, Tornetta P III. Stress examination of supination external rotation-type fibular fractures. J Bone Joint Surg [Am] 2004;86-A:2171–2178.

10. Gill JB, Risko T, Raducan V, Grimes JS, Schutt RC Jr. Comparison of manual and gravity stress radiographs for the evaluation of supination-external rotation fibular fractures. J Bone Joint Surg [Am] 2007;89-A:994–999.

11. Jenkinson RJ, Sanders DW, Macleod MD, Domonkos A, Lydestadt J. Intraop- erative diagnosis of syndesmosis injuries in external rotation ankle fractures. J Orthop Trauma 2005;19:604–609.

12. Schock HJ, Pinzur M, Manion L, Stover M. The use of gravity or manual-stress radiographs in the assessment of supination-external rotation fractures of the ankle.

J Bone Joint Surg [Br] 2007;89-A:1055–1059.

13. Koval KJ, Egol KA, Cheung Y, Goodwin DW, Spratt KF. Does a positive ankle stress test indicate the need for operative treatment after lateral malleolus fracture?

A preliminary report. J Orthop Trauma 2007;21:449–455.

14. Park SS, Kubiak EN, Egol KA, Kummer F, Koval KJ. Stress radiographs after ankle fracture: the effect of ankle position and deltoid ligament status on medial clear space measurements. J Orthop Trauma 2006;20:11–18.

15. Cheung Y, Perrich KD, Gui J, Koval KJ, Goodwin DW. MRI of isolated distal fib- ular fractures with widened medial clear space on stressed radiographs: which liga- ments are interrupted? AJR Am J Roentgenol 2009;192:W7–12.

16. Schuberth JM, Collman DR, Rush SM, Ford LA. Deltoid ligament integrity in lat- eral malleolar fractures: a comparative analysis of arthroscopic and radiographic assessments. J Foot Ankle Surg 2004;43:20–29.

17. Michelson JD, Varner KE, Checcone M. Diagnosing deltoid injury in ankle frac- tures: the gravity stress view. Clin Orthop Relat Res 2001;387:178–182.

18. Bossuyt PM, Reitsma JB, Bruns DE, et al. Standards for Reporting of Diagnostic Accuracy. Towards complete and accurate reporting of studies of diagnostic accuracy:

the STARD initiative. BMJ 2003;326:41–44.

19. Joy G, Patzakis MJ, Harvey JP Jr. Precise evaluation of the reduction of severe ankle fractures. J Bone Joint Surg [Am] 1974;56-A:979–993.

20. Sanders DW, Tieszer C, Corbett B. Canadian Orthopedic Trauma Society. Opera- tive versus nonoperative treatment of unstable lateral malleolar fractures: a rand- omized multicenter trial. J Orthop Trauma 2012;26:129–134.

21. Jaeschke R, Guyatt GH, Sackett DL. Users’ guides to the medical literature. III.

How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group.

JAMA 1994;271:703–707.

22. Landis JR, Koch GG. The measurement of observer agreement for categorical data.

Biometrics 1977;33:159–174.

23. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–310.

24. Sackett D, Straus S, Richardson W, Rosenberg W, Haynes R. Evidence-based medicine: How to practise and teach EBM. Second Ed. Edinburgh: Churchill Living- stone, 2000:67-68-93.

25. Weber M, Burmeister H, Flueckiger G, Krause FG. The use of weightbearing radiographs to assess the stability of supination-external rotation fractures of the ankle. Arch Orthop Trauma Surg 2010;130:693–698.

26. Hoshino CM, Nomoto EK, Norheim EP, Harris TG. Correlation of weightbearing radiographs and stability of stress positive ankle fractures. Foot Ankle Int 2012;33:92–98.