Abstract:Impact assessment
is usually based on direct surveys of treated individuals. The net impact
should be estimated observing untreated individuals which are very similar to
the treated group. However, the high cost of implementation does not allow to
use large samples, and therefore it is difficult to carry out statistically
significant comparison at the local level. The open access to databases which
were created for different purposes (e.g., tax, administrative, monitoring)
could overcome these limitations with many advantages. In this article we
present a validation exercise based on data records provided by the Italian law
whenever a change in an employment contract occurs, i.e. the so called
compulsory communications (COB). This was possible because of an exceptional
access to two different data sources, with the aim to assess the impact in
terms of employment of vocational training policies in Regione Piemonte
(northwest Italy), that is the COB database and a direct survey performed on
students one year after the course. We describe the major differences between
indicators calculated on survey data and indicators calculated on COBs.
Discrepancies are observed for 20% individuals in the sample analyzed. In
addition, the change of source distorts the results of the net impact
evaluation. In fact, both the determinants of the employment probability and
the importance of the net impact do significantly vary across sources. Whenever
an administrative database is used as a source for socio-economic analysis, it is
essential to be cautious and critic, validating its reliability by comparison
with different data sources. In the absence of such a preliminary validation
process, which should take place in close collaboration with regional and
national authorities managing the informational systems, researchers do not
only risk to uncritically accept information that provide systematic
distortions, but they prevent corrective procedures which are fundamental for
the development and improvement of the whole system.
Keywords: net
impact evaluation; validation; micro-data sources; survey; training policies