To begin evaluating the data your organization has been monitoring, each quarter your organization should produce a report, with the fourth quarter report being more robust than the other three.
Quarterly reports can be used to to stop and reflect, reevaluate your goals, strategy, and monitoring mechanisms, based on the rate of success in improvement from your initial baseline. Incorporating key staff (including program leaders, finance staff, operations staff, et al.) in this process is essential to effective intra-organization communication, to include those who support legal services to be informed of your organization’s priorities.
Match the evaluation with the goal
The evaluation process will be different for each type of goal. For example, for a substantive goal about access to work opportunities, you may want to consider the client’s work potential (in terms of salary, ability to obtain employment, discrimination, etc.); account for age, race, etc. when evaluating data, to account for possible factors outside of the quality of legal services provided that may affect one’s ability to obtain employment. You may find it useful to include a brief explanation of notable external developments affecting your success rates, independent of your organization’s ability to provide quality services.
Formal impact evaluation
In evaluating your impact of legal services, be mindful to identify big picture research questions that address the global impact of refugee legal aid. These questions are resource-intensive and difficult to internally analyze. These are not questions for lawyers or VLAs to address, but rather for external parties–academic or research institutions–with whom you collaborate to increase the transferability of your impact. These questions are resource-intensive and difficult to internally analyze.
Your monitoring mechanisms will play a role in your ability to determine a larger, global impact your organization may be having. Consider asking clients in follow up meetings questions such as “Did you gain any additional rights as a result of gaining status?” or “Have you informed other members of refugee community about work rights after meeting with us?”
You can conceptualize the formal impact of refugee legal aid in a few ways–the formal impact will be different in different contexts–but underpinning these hypotheses are:
1. Refugee legal aid makes it more likely that refugees will make wise decisions about their lives as a result of their access to valuable information about their rights.
This is the most vague, but also the most widely accepted.
2. Refugee legal aid increases the likelihood that refugees will be able to integrate into their new societies socially and/or economically.
There are several assumptions present here that could, and should, be explored. For example, it is widely believed that refugee legal aid will lead to status for clients, which will lead to integration. However, it must be explored whether status itself should be the starting point for accessing rights. It has not yet been studied whether having status leads to integration. Current literature does show that refugee legal aid increases the likelihood that a refugee will gain status but it’s unclear whether this status leads to an increased likelihood of economic integration. Of course, this could be hindered by other national processes, e.g. encampment policies, and any exploration would have to hold constant for age, duration, education level, and nationality, to isolate what role legal aid has played in helping refugees to access economic opportunity.
3. Refugee legal aid increases the likelihood that refugees will locate a durable solution in a shorter time frame.
These questions should form part of wider discussions that refugee legal aid providers should be having in conjunction with each other, to be addressed through partnerships with academic institutions.