Short-stay visa applications: a story of bias assessments

Short-stay visa applications: a story of bias assessments 

The Dutch Ministry of Foreign Affairs (MFA) has developed a tool in their internal analysis environment (Buitenlandse Zaken Analyse Omgeving, BAO) to support the assessment of short-stay visa applications (Kort Verblijf Visa, KVV). BAO is used in three ways: 

1. To check if there is information in the Visa Information System about the applicant, their reference, employer and/or travel documentation; 
2. To check if partners in the area of migration have information about the applicant or their reference; 
3. To compare the application to the current BAO profiles and local information. 

BAO assessment of applications 

With regard to the last point, the BAO uses profiles made based on historical information and results using a decision tree. Applications are categorised into three categories using these profiles: fast, regular and intensive. This label is intended as an advice to the person in charge of assessing the application, indicating how thorough their assessment should be. The three tracks have the following messages connected with them: 

1. Fast track: the application has little to no risk of abuse. The responsible person gets the following motivation for the this classification: “Application is part of a group with little to no risk of abuse.” 
2. Regular track: no information for a risk assessment. Assess using normal procedures. 
3. Intensive track: the application has an increased risk of abuse. The responsible person gets the following motivation for the this classification: “ATTENTION: Application is part of a group with a heightened risk of abuse.” 

Bias assessments 

During an internal Data Protection Impact Assessment (DPIA) of BAO, the question came up if it was legal to use the nationality of the person requesting the visa as part of the BAO. After external advice, the MFA concluded that this was allowed as long as the BAO did not contain bias. To find out if the BAO could contain bias, the MFA asked the Governmental IT Guild (Rijks ICT Gilde, RIG) to perform a bias assessment with the following questions (translated from Dutch)¹:

1. Which definition of bias is used in this assessment? 
2. Which types of bias can occur in the assessment of visa applications and within BAO? 
3. Is there bias in the construction and use of profiles? 
4. What is the most prominent conclusion with respect to the identified bias? 
5. Which steps should be taken to reduce the identified bias?  

After receiving the report of the RIG, the MFA decided to also obtain advice from SigmaRed. MFA did not accept the RIG report because, according to the MFA, the RIG went beyond the score of the questions and made assumptions which conflicted with the actual way BAO was being used. The scope of the SigmaRed report was: 

1. A detailed technical bias assessment of BAO; 
2. A detailed exploratory data analysis; and
3. Bias testing. 

The evaluation of bias in the final decision by a person or bias in the process was explicitly excluded from the scope.  

Conclusions of the bias reports 

Reading the reports of RIG and SigmaRed, you can see the differences in approach and scope. Since their focus is different, it is worth recapping the conclusions of both and comparing them in the next section. 

RIG conclusions 

To define bias, the RIG determined that the most important metric was that the chance of a successful application ending up on the intensive track should be the same for groups with different demographics. This is called the False Positive Rate.  

The report highlights a number of types of bias which can occur in BAO, specifically historical bias in the collected data, confirmation bias in the modelling of BAO, feedback bias in the usage of BAO and automation bias in the use of BAO for individual applications. Only one of these four is related to the technical construction of the BAO, the others are related to the process. RIG notes that with the data available to them, they can’t say much about the majority of these sources of bias. 

The report states that there are bias discrepancies based on nationality and limited bias discrepancies based on gender. This means that accepted applicants with some nationalities have a significantly higher chance of being placed in the intensive track than other nationalities. However, the RIG does not provide any conclusions of the effect of this bias on the final decision, due to the lack of suitable data. 

RIG gives advice for short-term and medium-term measures to reduce the identified bias. In the short term, the RIG advices terminating the use of profile scores, limiting the intensive track as a possible outcome when certain information is not available and more. In the medium term, they advice an evaluation of the BAO using the Fundamental Rights and Algorithm Impact Assessment (FRAIA), improving the management of BAO, applying technical bias-mitigation methods and more. 

SigmaRed conclusions 

SigmaRed performed statistical bias tests on the dataset of visa applications for a set of protected attributes. They found no disproportionate bias based on age, marital status or gender. Disproportionate bias would be if the bias for a certain attribute would fall outside a range of uncertainty. For one nationality, SigmaRed found a disproportionate bias.  

The report states that, since the BAO does not generate predictions but instead categorises applications, metrics like the False Positive Rate are irrelevant. These suppose a binary outcome, which they consider incompatible with the function of BAO. 

The analysis of the visa data indicates a notable correlation between the BAO profile and the visa outcomes, even after correction. SigmaRed does indicate they were not able to exclude the effect of unobserved factors. 

SigmaRed recommends that the MFA performs more research to determine the effect of the BAO tract on the decision for a visa. They also recommend establishing a monitoring system for BAO to detect shifts in data patterns, performance or bias indicators. 

Comparison of conclusions 

The first thing to notice when comparing these two conclusions is that the scope of SigmaRed is much more limited. However, the RIG and SigmaRed do reach some similar conclusions. The most important conclusion: it is hard to determine the bias of the BAO system, since the decisions of the people using BAO are taken as the ground truth. What does this mean? The MFA claims that the different tracks are not risk categories, but just an advice to the assessing person. However, both RIG and SigmaRed suspect that merely being put in the intense track already lowers your chances of getting a visa. They both indicate that this is hard to confirm. Both reports also advice that the management and monitoring of BAO needs to be improved. The two conclusions do not agree on everything. Notable, SigmaRed completely disagrees with the decision of the RIG to use the False Positive Rate as a measure of bias. 

Take-aways 

These two reports show that there is not one way of conducting a bias assessment. In my opinion, the approach of the RIG is more useful than the approach of SigmaRed since it concerns the entire decision making process and not just the BAO. In the end, the decision on the visa application should be free of bias, not just the BAO. However, both reports also show the difficulties with checking for bias in this type of system where the previous results are used as the ground truth for assessing how well the system works.  

¹ Antwoord op vragen het lid Dekker-Abdulaziz over het bericht over het visumbeleid door het ministerie van Buitenlandse Zaken | Tweede Kamer der Staten-Generaal