A Belief Rule Base Expert System for staging Non-Small Cell Lung Cancer under Uncertainty

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

A Belief Rule Base Expert System for staging Non-Small Cell Lung Cancer under Uncertainty

Munmun Biswas

Department of Computer Science and Engineering

BGC Trust University Bangladesh Bidyanagar, Chandanaish, Bangladesh

munmunbiswas29@yahoo.com

Mohammad Shahadat Hossain Department of Computer Science and

Engineering University of Chittagong University-4331, Bangladesh

hossain_ms@cu.ac.bd

Mr. Salah Uddin Chowdhury Department of Computer Science and

Engineering

BGC Trust University Bangladesh Bidyanagar, Chandanaish, Bangladesh

schowdhury_cse@yahoo.com

Karl Andersson Pervasive and Mobile Computing

Laboratory

Luleå University of Technology SE-931 87 Skellefteå, Sweden

karl.andersson@ltu.se

Nazmun Nahar

Department of Computer Science and Engineering

BGC Trust University Bangladesh Bidyanagar, Chandanaish, Bangladesh

nazmun4@gmail.com

Abstract— Non small cell Lung cancer (NSCLC) is one of the most well-known types of Lung cancer which is reason for cancer related demise in Bangladesh. The early detection stage of NSCLC is required for improving the survival rate by taking proper decision for surgery and radiotherapy. The most common factors for staging NSCLC are age, tumor size, lymph node distance, Metastasis and Co morbidity. Moreover, physicians’

diagnosis is unable to give more reliable outcome due to some uncertainty such as ignorance, incompleteness, vagueness, randomness, imprecision. Belief Rule Base Expert System (BRBES) is fit to deal with above mentioned uncertainty by applying both Belief Rule base and Evidential Reasoning approach .Therefore, this paper represents the architecture, development and interface for staging NSCLC by incorporating belief rule base as well as evidential reasoning with the capability of handling uncertainty. At last, a comparative analysis is added which indicate that the outcomes of proposed expert system is more reliable and efficient than the outcomes generated from traditional human expert as well as Support Vector Machine (SVM) or Fuzzy Rule Base Expert System (FRBES).

Keywords—Non-Small Cell Lung Cancer (NSCLC), Expert System, Uncertainty, Belief rules base, Evidential Reasoning.

I. INTRODUCTION

Lung Cancer is considered as one of the primary cancer everywhere throughout the world. It is a disease in which tumor is formed by abnormal and uncontrolled cell growth in certain lung cell. It is a common reason for cancer related demise in both man and women in Bangladesh. The statistics of Dhaka Medical College dept. of Radiotherapy, 21% male in suffer from lung cancer in Bangladesh [1].

Lung cancer is generally separated into two sorts, one is Small cell lung cancer (SCLC) and other is Non-small cell lung cancer (NSCLC).The rate of occurrence of NSCLC is 85% which is much higher than the occurrence of SCLC [2].However, Surgery and Radiotherapy provide better result to detect at an early stage. NSCLC can spread through the lymphatic and the blood. An exact determination is essential to decide the stage of NSCLC. Doctors require the assessment for staging NSCLC to talk about survival rate of a patient. The stage of NSCLC is resolved from 3 key pieces

of information including size of tumor, spread to close-by lymph nodes and spread to distant organs like adrenal glands, brain, bones, liver, kidneys or other lung which are considered as Metastasis [3].Age is another factor for staging NSCLC. The patient above 60 years is more affected by NSCLC than the patient under 60 years. [4]. Co- morbidity is additionally a factor to decide stage of NSCLC, for example, an older patients can present other diseases such as heart failure, cerebrovascular diseases, chronic obstructive lung disease and myocardial infarction. These types of Co morbidity effect on survival [5].

A physician usually determines the stage of NSCLC from symptoms expressed by a patient and the result of a physical exam such as biopsy and imaging tests of the patient [5].

However, uncertainty exists for observer error and scanning error. For this reason, it is not able to infer 100% accurate result stage of NSCLC. Poor image quality, incorrect patient positioning and movement are the examples of uncertainty of tests [6]. In many cases, due to such uncertainty a physician cannot able to assess the diseases accurately. So uncertainty requires addressing carefully to get an accurate assessment result.

Therefore, the manual way cannot achieve an error-free outcome and better algorithmic answer for staging NSCLC.

Besides, an expert system is a proper tool which ready to emulate decision to detect the stage of NSCLC. An expert system generally consists of two parts, one is knowledge base and other is inference engine. Knowledge representation schema of an expert system acquires uncertain clinical knowledge and inference engine should have robust algorithm which able to handle different kinds of uncertainty as mentioned before. So in this pathway, a belief rule base expert system (BRBES) has been considered to handle uncertain knowledge by belief rule base and the evidential reasoning is performed in inference engine.

In this paper, Section II presents the literature review, section III introduces the belief rule base approach, section IV discusses the system design, Architecture and

i

implementation of BRBS and section V presents the result and discussion, while section VI concludes the paper.

II. LITERATURE REVIEW

Some review of expert system for diagnosis of non-small cell lung cancer has been described in this section.

A Fuzzy rule based Expert system for diagnosis lung cancer was designed by[7].The Fuzzy logic is capable to handle some sorts of vulnerability such as vagueness, ambiguity but not able to catch incompleteness and ignorance which is also required for assessing lung cancer from various symptom and risk factor.

A Lung Cancer risk prediction system was also implemented for assessing risk level of lung cancer. K- means cluster algorithm was used to develop this system.

Hence, the system cannot be able to capture uncertainty from sign and symptom of a patient [8].

Support vector machine and Logistic Regression was used for classifying the survival rate of patient suffered from lung cancer [9].However, for the absence of straight forwardness, SVM cannot deal with uncertainty between sign and symptoms of a patient.

Radionics was explored to predict survival time of Non- Small cell Lung Cancer (NSCLC) by extracting the quantifiable data from Computer Tomography image [10].

A Meta-analysis was used in expression of gene signature to predict survival of stage I Non-small cell lung cancer [11].

However, above methods cannot find the relationship between signs and symptoms of the patient in a considerable way. Therefore, proper prediction system is required for identifying the stage of NSCLC from pathological data [12].

Predicting the stage of NSCLC is a complex problem and so an expert system is essential for handling all sorts of uncertainty. A belief rule base expert system which is designed and developed from Belief Rule-Based Inference Methodology using the Evidential Reasoning (RIMER) approach[13][14][25-37] is used in this paper. All sorts of uncertainty are able to address by this approach. A Belief rule base expert system consists of belief rule base and evidential reasoning. In belief rule base is a belief degree is embedded to each consequence of a rule. Inference of each rule is performed using evidential reasoning which is able to handle different types and degree of uncertainty from pathological data.

III. BELIEF RULE BASE APPROACH

In this part, a knowledge base and inference engine are presented according to the belief rule base approach. These two portions are explained by a number of procedures.

A. BRB knowledge represantation

A belief rule base consists of a number of rules which acts as the knowledge base of an expert system. A belief rule is an expansion of customary IF-THEN rule represents the knowledge base. In a belief rule, the antecedent part contains antecedent attributes with its referential values whereas the consequent part contains consequent attributes with its possible referential values. Belief Rule Base consists of some knowledge representation criteria such as rule weight, antecedent attribute weight and belief degree [14].The belief rule can be formed by the following

equation-

:

( ) ∧ ( ) … . .∧

{( ), ( ) … ( )}

ℎ ∑ ≤ 1

ℎ ℎ

ℎ , ,… ,

(1)

Equation (1) indicates S1, S2, S3 …STk as the antecedent attributes where (i = 1, … … , Tk, k = 1,… …, L) represents one of the referential values of the i^th antecedent attribute Si in the kth rule. In the belief rule, Pj (j=1,2…N) is j^th referential value of consequent with belief degree jk (j

= 1, …… , N,k = 1,…… , L) where L indicates rule number and N is the number of referential values .A rule k is said to be finished if

∑ = 1

A case of belief rule in the area of Non-small cell lung cancer given below

:

ℎ ℎ ℎ

ℎ

ℎ

( ℎ ,0.90), ( , 0.05), ( , 0.05) (2)

At the above equation (2) the referential values of Non-small cell lung cancer consequent is in the distribution {(High, (0.9)), (Medium, (0.05)), (Low, (0.05))}. It is observed that, the assessment is complete as the sum of belief degree is (0.9+0.05+0.05) =1.

B. BRB Inference scheme

The inference scheme of BRB is made of four consecutive parts which is described below in this section.

• Input Transformation

In this part, input of the antecedent attribute Si is disseminated through its diverse referential values [14].

The following equation (3) states the estimation of the input value.

Z (Si,εi) = {(Aij, αij), j = 1, … , ji}, i = 1, … … , Tk (3) At equation (3) Z indicates the measurement on degree of belief which is assigned as input value to the antecedent attribute. The j^th referential value Aij (ith value) of the input Si is distributed in the belief degree αij with αij≥0 and

∑

Here, the Linguistic terms 'High', 'Medium' and 'Low’ use as an input value of the antecedent attribute. Initial inputs are gathered from the patients and experts regarding the Non small lung cancer. A matching degree of belief εi is then assigned to each of the linguistic term according to the expert judgment. Later, this matching degree of belief is distributed through different referential values according to belief degree aij. The following equation (4) and (5) illustrate the process input transformation.

ℎ ℎ = , (4)

= (1 ), = 1

ℎ ℎ = , (5)

= (1 ), = 1

The utility λij of Aij is assigned as λi3=1 for “High”

referential value, λi2=0.5 for “Medium” referential value and λi1=0 for “Low” referential value.

The input value of antecedent attributes is transformed by above mentioned Eq. (4) and Eq. (5) along with expert belief degree of each input into referential values of antecedent attributes is shown in Table 1.For instance, the input value of “Age” attribute of NSCLC staging is .8 which is transformed into referential values High (0.4), Medium (0.6) and Low (0.0) by applying Eq. (4) and Eq. (5).

• Estimation of Rule activation weight

At the steps of rule activation weight estimation, the matching degree αk is required which is determined by the Eq.(6)[25].

= ∏ =

,…, { } (6) In the above eq (6) is the individual degree and is the weight of i-th attribute for k-th rule. The activation weight of a rule is determined by the matching degrees which are assigned to the referential values of the antecedent attributes.

Now the rule activation weight ωk is determined by employing Eq. (7) [14].

=_∑ = ^∏

∑ ∏

relative weight of the k-th rule.

• Belief updating

It may be observed that input estimation of any of the factors related with NSCLC framework may not be available. In that case, the initial consequent belief degrees need to be updated to deal such type of uncertainty. The initial belief is modified by Eq. (8) [15] [16].

β = β ^{∑ ( , ) ∑}

∑ ( , ) (8) Where

( , ) = 1, ( = 1, … , ) 0, ℎ

• Combination of Rules using ER

The inference step of this expert system uses Evidential Reasoning approach to aggregate the all packet antecedents of the L rules. All things considered, ER approach can be utilized in recursive or analytical way to finish the aggregation process. The analytical ER approach is considered to apply in this research to diminish computational complexity [17] [18]. The final consequent referential value is obtained with conclusion C(Y) by using following Eq. (9).

C(Y) = {(Pj,βj), j=1,2….N} (9) where βj is the belief degree regarding one of the consequent’s referential values Pj. The belief degree βj is acquired by applying analytical ER algorithm which is as Eq.

(10) [16] [18].

βj = ^{∏ ∑}_∏ ^{∏ ∑} (10)

With

μ = ω β 1 ω β (N 1)

1 ω β

The resulting output of ER is represented as {(P1, β1), (P2,

β2), (P3, β3)……. (PN, βN)}, where βj denotes the final belief degree of the jth consequent referential value Pj . This outcome is changed into numerical or crisp value by apportioning utility score to each referential value of consequent attribute [14].

( ^∗) = ∑ u(P )β (11) Where H (A^*) refers the expected numerical score which is obtained by the utility score u (Pj) of each referential value.

IV.BRBES FOR STAGING NON SMALL CELL LUNG CANCER

This section represents the architecture and implementation techniques of the Belief Rule-Based Expert System (BRBES) for staging Non-small cell lung cancer (NSCLC).

A. Architecture and implementation

System architecture can be characterized how its component are organized. In this work BRBES are presented by three- layer architecture where one is User Interface layer and other two are application layer and data management layer as show in Fig.1.

User Interface layer: The interface layer has been developed to acquire the value of antecedent attributes from physician or patients and show the result of NSCLC suspicion

Fig.1: BRBES System Architecture

User Interface Layer

Application Layer

User

User Interface

Request

Respons

Data Access Inference Engine

Database System Clinical Facts

Knowledge Base

The integration of various web programming such as HTML, and CSS has been used to develop this interface.

Application layer: The application layer contains inference procedures along with training module. Input transformation, rule activation weight, belief update and rule aggregation is calculated by using evidential reasoning (ER). PHP has been utilized to build up this layer.

Data Management Layer: In this layer knowledge base is stored by utilizing MySQL. MySQL is a relational database which is user friendly and use as back-end to store and manipulates initial BRB.

B. BRBES Knowledge base

A belief rule base (BRB) has been designed by a BRB tree that is appeared in Fig.2 which identifies antecedent and consequent attributes to build knowledge base of BRBES.

Fig.2: BRB Framework for staging NSCLC

The leaf node of tree represents the antecedent attributes and root node represents consequent attribute [19]. The five antecedents attribute which represent as factors for staging are identified and verified by consulting with physician of various hospitals located in Chittagong city of Bangladesh.

This BRB consists of 162 rules which obtain from five antecedent attributes where A1, A2, A3, A5 antecedent attributes has three referential values and A4 has two referential values. The rule numbers are determined by utilizing Eq. 3. Each rule of the BRB has given rule weight

"1" while every antecedent attribute’s weight is considered as "1". A case of such rule is represented in Eq. 2.

Table 1: The initial Rules of the BRB.

The instance of belief rule shows elaborately in Table 1.

C. ER Approach use in Inference Engine

Evidential reasoning(ER) algorithm is used for Inference Engine [20] which is mentioned in section 3.2.The technique of inference engine completes in five steps.

Firstly collects factors from both patients and physicians.

Secondly apply Eq (4) and Eq(5) to convert data into matching degree by performing input transformation.

Thirdly calculate activation weight for each rule applying Eq(6) and Eq(7).Fourthly update belief degree by employing Eq(8) for the ignorance. Finally aggregation of rule is calculated by applying Eq (9), Eq (10) and Eq(11).

D. System Interface

A system interface is used as a medium to interact between user and system. The interface of BRB expert System for staging NSCLC is shown in Fig.3, where the input value of five antecedent attributes are gathered from both patients and physicians.

Fig 3: BRBES for staging Non-small Cell Lung Cancer

For example if the value of A1 (Age) is .8 .Then A1 is converted into its three referential value which is (High, .4), (Medium, .6) and (Low, 0).This value is obtained from input transformation by using Eq(4),Eq(5).

The belief degree of each referential values is calculated by applying Eq(6),Eq(7) and Eq(8)and shown as (High,.953),(Medium,.019),(Low,019).

All the value is aggregated into a crisp value by using Eq(9),Eq(10) and Eq(11) and calculated as .9725.The assessment result of system measure that the patient is suffered from Stage IV NSCLC.

V. RESULT AND DISCUSSION

Patient dataset collected from various hospitals of Chittagong Region to assess NSCLC. Table 2 presents only data of 5 patients out of 120patients. Column 6 shows the Bench Mark results which is gathered from the laboratory investigation of 50 patients.

Table 2: The Result of BRBES, Expert, SVM, and FRBES from dataset

SL No BRBES Expert Opinion

SVM FRBES Suspicion(Outcome)

1 30.16 34.89 21.45 26.1 0

2 40.32 42.56 31.00 43.41 0

3 52.15 54.34 50.62 53.82 1

4 72.93 73.03 68.23 66.93 1

5 82.56 84.21 78.21 80.62 1

Rule ID

R.

W

IF Then NSCLC A1 A2 A3 A4 A5 H M L R1 1 H H H H H 1 0 0 R2 1 H H H H M 0.9 0.1 0 R162 1 L L L L L 0 0 1

Non Small Cell Lung Cancer

A1: Age A2: Tumor size A3: Distance from Lymph node

A4: Metastasis A5: Co morbidity A3

NSCLC

A1

A2 A4

A5

The Bench Mark result is “1”, if the patient’s Expert opinion or observation of other system on the NSCLC level is greater than 50 and otherwise the bench mark is “0”.The result “1” shows for suspicion and “0” out of suspicion for NSCLC. Column 7 demonstrates the assessment of NSCLC by using the BRBES. MatLab Environment is used to create outcomes for similar information by utilizing SVM and FRBES which are recorded in Column 9 and Column 10.Other column of table 5 displays the information of 10 patients out of 120.

The Receiver operating characteristic (ROC) curves which are successfully utilized for accurate analysis of symptomatic test [21].The ROC can be used to compare BRBES against Expert opinion also with SVM and FRBES on the basic of Bench Mark data. The accuracy of output for staging NSCLC can be measured by the area under curve (AUC) [21] [22] [23].

Fig. 4: Evaluation comparison among BRBES, Expert opinion and SVM using ROC curve.

ROC curves which is plotted for BRBES, Expert and SVM is shown in Fig 5. The blue line of ROC curve is generated by the assessment result of BRBES with AUC of 0.996 associate with 95% confidence interval .876-.998.

Fig. 5: Evaluation comparison among BRBES, Expert opinion and FRBRS using ROC curve

In Fig 6 the pink line is shown the result of FRBES associated with AUC of .920 and 95% confidence interval .810-.940.SPSS statistics 23 is used to generate the ROC curve and also measure the value of AUC.

The green line is associated against the expert opinion with AUC of .986 along with 95% confidence interval .856-.992, and the orange line is also obtained for the result of SVM and its AUC is about .911 with 95% interval .790-.931.

The Table 3 summarizing the result from Fig. 4 and Fig. 5 associated with BRBES, Expert opinion, FRBES and SVM.

Table 3: AUC Value comparison among four systems Test outcome Area Asymptotic 95% confidence interval

Lower bound Upper bound

BRBS .996 .876 .998

EXPERT .986 .856 .992

FRBES .920 .810 .940

SVM .911 .790 .931

From Table 3 shows the AUC of expert opinion is higher than the FRBES and SVM but less than BRBES.

Therefore, the result form BRBES for staging NSCLC is more reliable than other three systems. The reason for that during the conversation with patient and physician is not awarded about the uncertainty issues related to the factor for staging NSCLC. Evidential reasoning of BRBES is ready to capture various types of uncertainty. BRBES inference system consists of rule weight, attribute weight and belief degree updates to handle the multidimensional problem.

Actually, FRBES is ready to catch some types of uncertainty but not able to handle uncertainty such as imprecision, vagueness, ambiguity.

And other side SVM learning works on limited number of parameters. So it reduces the performance of the system when it works on multiple learning parameters [24].

Therefore the result of BRBES is much more reliable than other two methods FRBES and SVM.

VI. CONCLUSION

According to the recent statistics, the death rate of Non- small cell lung cancer increases rapidly all over the world.

Therefore, an efficient system to diagnose the stage of Non- small Cell Lung cancer is essential for the patient who is suffering from NSCLC. This Research exhibits a system which is developed by utilizing Belief Rule Base Expert System (BRBES) for staging the Non-Small Cell Lung Cancer (NSCLC) from various factors of pathological data.

The outcome obtained from BRBES is more reliable than Expert opinion, Fuzzy Rule based Expert System (FRBES) and Support Vector Machine (SVM).For that reason, the application of BRBES can be able to provide a decision making platform to physicians and will help patient to get better treatment in advance. In future, the system would be considered to improve the optimal learning capability of BRBES by training all knowledge representation parameters namely rule weight, attributes weight, degree of belief in order to get more efficient results.

R^EFERENCES

[1] R. F. Mukti, P.D. Samadder, A. A. Emran, F. Ahmed, I. B. Imran, A.

Malaker, S. Yeasmin, “Score Based Risk Assessment of Lung Cancer and its Evaluation for Bangladeshi People,” Asian Pacific Journal of Cancer Prevention,Vol15, 2014.

[2] Genetic Home Reference, Lung Cancer. U.S., National Library of Medicine. https://ghr.nlm.nih.gov/condition/lung-cancer

[3] Non-small cell lung cancer stages. American Cancer Society.

https://www.cancer.org/cancer/non-small-cell-lung-cancer/detection- diagnosis-staging/staging.html

[4] F. Yang, H. Chen, J. Xiang, Y. Zhang, J. Zhou, H. Hu, J. Zhang, X.

Luo, “Relationship between tumor size and disease stage in non-small cell lung cancer”. BMC Cancer, 10:474, September 2010.

[5] A. M. Green, E. Jakobsen, “The direct and indirect impact of comorbidity on the survival of patients with non-small cell lung cancer: a combination of survival, staging and resection models with missing measurements in covariates”. BMJ Open 2014; 4:e003846. Doi:

10.1136/bmjopen-2013-003846

[6] M.S. Hossain, F. Ahmed, F.T. Johora , K..Andersson ,”A believe rule based expert system to assess brocnchiolitis suspicion from sign and symptom under uncertainty” ,J Med Sys (2017),41:43.

[7] F.V. Farahani, M.H. F. Zarandi, A. Ahmadi, (2015),” Fuzzy rule based expert system for diagnosis of lung cancer”, DOI: 10.1109/NAFIPS- WConSC.2015.7284206.

[8] J. Huysmans, K. Dejaeger, C. Mues, J. Vanthienen, and B. Baesens,

“An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models,” Decis.Support Syst., 51(1):141–154, 2011.

[9] A. Hazra, N. Bera, A. Mandal, “Predicting Lung Cancer Survivability using SVM and Logistic Regression Algorithms”. International Journal of Computer Applications, Volume 174 – No.2, September 2017.

[10] S .H .Hawkins, J. N. Korecki, Y .Balagurunathan, Y. Gu, V. Kumar, S.

Basu ,L.O. Hall,” Predicting Outcomes of Non-Small Cell Lung Cancer Using CT Image Features”,+IEEE Access 2:1418-1426, DOI: 10.1109/ACCESS.2014.2373335, January 2014.

[11] Y. Lu, W. Lemon, P.Y. Liu, Y. Yi, C. Morrison, P. Yang,, Z .Sun, J. Szoke , W.L. Gerald, M. Watson, R. Govindan, M.. You, “A Gene Expression Signature Predicts Survival of Patients with Stage I Non-Small Cell Lung Cancer”, PLoS Medicine, December 2006, Volume 3, Issue 12, e467

[12] R.O. Dillman,S.L.Seagren, K.J. Propert , J. Guerra,, W.L.

Eaton, M.C. Perry , R.W. Carey, E.F. Frei, M.R. Green,” A randomized trial of induction chemotherapy plus high-dose radiation versus radiation alone in stage III non-small-cell lung cancer”,PMID: 2169587,DOI:

10.1056/NEJM199010043231403.

[13] M.S. Hossain, M.A.Hasan, M. Uddin, M.M. Islam, R. Mustafa , “A belief rule based expert system to assess Lung Cancer under uncertainty”,18^th International Conference on Computer and Information Technology(ICCIT),21-23 December,2015.

[14] J. B. Yang, J. Liu, J. Wang, H. S. Sii and H. W. Wang, “Belief rule- base inference methodology using the evidential reasoning approach - RIMER.,” IEEE Transactions on Systems Man and Cybernetics Part A- Systems and Human, 36, 266-285, 2006.

[15] D. L. Xu, J.B Yang, Y.M. Wang, "The evidential reasoning approach for multi-attribute decision analysis under interval uncertainty," European Journal of Operational Research, 174.3 (2006): 1914-1943.

[16] C. Angulo, J. Cabestany, P. Rodríguez, M. Batlle, , A. González, and S.D. Campos, “Fuzzy expert system for the detection of episodes of poor water quality through continuous measurement”, Expert Systems with Applications, 39(1), 1011–1020, 2012.

[17] G.L. Kong, D.L. Xu, J.B. Yang, “An evidence-adaptive belief rule- based decision support system for clinical risk assessment in emergency care”, In: 23 rd European Conference on Operational Research, Bonn, Germany, 2009.

[18] J. B. Yang, P. Sen, “A general multi-level evaluation process for hybrid MADM with uncertainty”, Systems, Man and Cybernetics, IEEE Transactions on, 24, 1458-1473, 1994

[19] D. L. Xu, J. Liu, J..B. Yang, G.P. Liu, J. Wang, I. Jenkinson, J. Ren,

“Inference and learning methodology of belief-rule-based expert system for pipeline leak detection”, Expert System Application, 32(1):103–113 [20] J. .B. Yang, "Rule and utility based evidential reasoning approach for multi attribute decision analysis under uncertainties," European journal of operational research, 131.1 (2001): 31-61.

[21] R.Body, “Clinical decision rules to enable exclusion of acute coronary syndromes in Emergency Department patients with chest pain”, Turk J Emerg Med, 2018 Sep; 18(3): 94–99.

[22] J.A .Hanley, "The robustness of the" binormal" assumptions used in fitting ROC curves," Medical decision making, 1988 Jul-Sep; 8(3):197-203 [23] E. R. DeLong, D. M. DeLong, D. L. C. Pearson, “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”, Biometrics, Vol. 44, No. 3 (Sep., 1988), pp. 837-845

[24] Y. M. Wang, J. B. Yang, D .L. Yang (2006), “Environmental impact assessment using the evidential reasoning approach”, Eur J Oper Res,Vol 174,No.3(Nov,2006),pp.1885-1913.

[25] M. S. Hossain, I. B. Habib, and K. Andersson, “A belief rule based expert system to diagnose dengue fever under uncertainty,” in 2017 Computing Conference, July 2017, pp. 179–186

.

[26] M. S. Hossain, M. A. Haque, R. Mustafa, R. Karim, H. Dey, and M. Yousuf, “An expert system to assist the diagnosis of ischemic heart disease,” Int J Integr Care, 2016

[27] M. S. Hossain, S. Rahaman, R. Mustafa, K. Andersson, “A belief rule-based expert system to assess suspicion of acute coronary syndrome (acs) under uncertainty,” Soft Computing, vol. 22, no. 22, pp. 7571–7586, 2018.

[28] R. Karim, K. Andersson, M. S. Hossain, M. J. Uddin, M. P. Meah, “A belief rule based expert system to assess clinical bronchopneumonia suspicion,” in 2016 Future Technologies Conference (FTC). IEEE, 2016, pp. 655–660.

[29] M. S. Hossain, K. Andersson, S. Naznin, “A belief rule basedexpert system to diagnose measles under uncertainty,” in World Congress in Computer Science, Computer Engineering, and Applied Computing (WORLDCOMP’15): The 2015 International Conference on Health Informatics and Medical Systems 27/07/2015-30/07/2015. CSREA, 2015, pp. 17–23.

[30] M. S. Hossain, F. Ahmed, K. Andersson, "A belief rule based expert system to assess tuberculosis under uncertainty." Journal of medical systems 41.3 (2017): 43

[31] S. Rahaman, M. S. Hossain, "A belief rule based clinical decision support system to assess suspicion of heart failure from signs, symptoms and risk factors." 2013 International Conference on Informatics, Electronics and Vision (ICIEV). IEEE, 2013.

[32] M. S. Hossain., M.S.Khalid, S. Akter, S.Dey, “ A belief rule-based expert system to diagnose influenza”. In 2014 9Th international forum on strategic technology (IFOST) (pp. 113-116). IEEE.

[33] M.S.Hossain, A.A Monrat, M.R. Hasan, M.Karim, T.A.Bhuiyan, M.S.Khalid, “A belief rule-based expert system to assess mental disorder under uncertainty”. In 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV) (pp. 1089-1094). IEEE.

[34] M. S.Hossain, P.O.Zander, M.S.Kamal, L.Chowdhury ,“Belief rule based expert systems for evaluation of e government: a case study”. Expert Systems, 32(5), 563-577.

[35] S.T Alharbi, M. S. Hossain, A. A. Monrat, “A belief rule based expert system to assess autism under uncertainty”. In Proceedings of the World Congress on Engineering and Computer Science (Vol. 1).

[36] M. S.Hossain, E. Hossain, S. Khalid, M.A.Haque, “A Belief Rule Based (BRB) Decision Support System to Assess Clinical Asthma Suspicion”. In Scandinavian Conference on Health Informatics; August 22;

2014; Grimstad; Norway (No. 102, pp. 83-89). Linköping University Electronic Press.

[37] M. S. Hossain, S. Akter, S. Rahaman “A Belief Rule Based Expert System to Assess Meditation.” In 2015 International Conference on Computational Science and Computational Intelligence (CSCI) (pp. 829- 832). IEEE.