School supervision: A tool for standardization or for equity, A De Grauwe

Tags: supervision, developing countries, IIPE, education system, decentralization, Finland, government, Scandinavian Journal of Educational Research, Paris, supervision services, central control model, national curriculum, disparities, external supervision, Comparative education, educational planning, education policies, New Zealand schools, Oxford University Press, International Journal of Leadership in Education, Educational Research, Department for international development, Multi Serve Education Trust, education policy, Hellenic Ministry of Education, NZARE Conference, Educational leadership, teacher professionalism, University Department of Education, educational reform initiatives, public education system, control model, discussion with teachers, School supervision, internal school, OFSTED, evaluation procedures, national performance indicators, Anton De Grauwe Working, Education accountability, Journal for Critical Education Policy Studies, Southern African Education, Gabriel Carron, Anton De Grauwe Draft, De Grauwe, British Educational Research Journal
Content: DIRECTIONS IN EDUCATIONAL PLANNING: SYMPOSIUM TO HONOUR THE WORK OF FRANЗOISE CAILLODS LES ORIENTATIONS DE LA PLANIFICATION DE L'ЙDUCATION: SYMPOSIUM EN L'HONNEUR DU TRAVAIL ACCOMPLI PAR FRANЗOISE CAILLODS Thursday 3 July ­ Friday 4 July 2008 Jeudi 3 juillet ­ Vendredi 4 juillet 2008 School supervision: a tool for standardization or for equity? Anton De Grauwe Working document Document de travail International Institute for Educational Planning Institut international pour la planification de l'йducation
School supervision: a tool for standardization or for equity?1 Anton De Grauwe Draft paper presented to the symposium on "Directions in educational planning", IIEP, 3-4 July It is well known that the US Declaration of Independence holds it as a self-evident truth "that all men are created equal". While it is a sentiment and a conviction shared by many policy-makers and one which has formed the basis of revolutions throughout ages, it is equally self-evident that all societies are still characterized by more or less significant inequalities. The debate about the role of education in the construction or deconstruction of inequalities is longstanding and complex. The differences in opinion between those who interpret the schooling system as a force for reproduction of inequalities and those who see schooling as a force for social mobility and as a fundamental driver towards a more equal society undoubtedly reflect genuine differences in analysis. These differences are also nourished by the diversity in education policies, some of which indeed reconfirm and strengthen existing disparities while others aim at improving the social position and educational status of the most disadvantaged. The position of Franзoise Caillods in this debate is well known. She recognizes without any naivety the reasons for the continued existence of social and economic disparities, which are evidently linked to the distribution of political power but also to the even more insidious "culture of poverty". But she believes strongly in education as a tool to break social barriers and glass ceilings and as a source for empowerment. For education to play this role, education policies need to be guided by a consistent concern for equity. Such a concern for equity implies several strategies, for instance the need to adapt the content, teaching language and methods to the characteristics of disadvantaged groups. More specifically within the field of educational planning and management, it implies that ministries of education give priority in the distribution of their resources to those schools and those social groups who are now most disadvantaged. This is even more true for developing countries where governments have scarce resources than for the most developed countries with a relative abundance of funds. Indeed, precisely when governments have few resources to distribute, it is their duty to ensure that these resources are directed firstly towards those most in need, who seldom have anywhere else to go for support. The better-off groups tend to have access to other channels for financial and even technical support. However, precisely in the least developed countries, the constraints to equity-focussed policies may be more difficult to surmount for both technical and political reasons, to which we will return in a concluding section of this paper. Somewhat unfortunately, when discussing needs-based resource allocation, the term "resources" have almost always been interpreted as "financial resources" only. This is unfortunate for three reasons. Firstly, in the least developed countries, the financial resources which the government makes available to schools, are mostly used for salaries and the amount which is flexible and could be directed towards the most needy schools is generally insufficient to make much of a difference. Secondly, simply making more finances available to these schools does not guarantee that these funds will be used to improve school functioning and quality. This is not only a matter of possible mismanagement or lack of transparency, though case-studies on financial 1 This text is a draft version. It is based partly on a contribution prepared for the Global Monitoring Report 2009. Some references are still missing in this draft. This version should not be copied neither distributed. 2
decentralization have highlighted these risks; it is also a question if school leaders have the necessary insights and information to make the best judgments on how to use school funds. Thirdly, while ministries may not necessarily distribute many funds to schools, they have many other interactions with schools, some of which have potentially a greater impact than the scarce funds. One eminent example is school supervision. Indeed, in nearly every country, the ministry of education has set up since the creation of the public education system a supervision service, the main task of which is to control and offer support to schools and teachers. It is somewhat surprising that, within the literature and research on equity-focussed education policies, very little attention has been given to the question how the organization of school supervision and support can be infused with a concern for disparities. Supervision policies in many countries have been equally inattentive to this issue. This article will, in a first section, examine different models of school supervision and, in a second section, it will ask the question what attention these models pay to differences and disparities between schools and what impact they have on these disparities. The identification of these models or "ideal types" of school supervision is based on previous research work done either within IIEP, much of it in collaboration with Gabriel Carron (Carron and De Grauwe, 1997; De Grauwe and Carron, 2006) or individually (see De Grauwe, 2003 and De Grauwe, 2006). Identifying school supervision models Supervision services have in almost all countries accompanied the setting up of a public education system, which itself formed an integral part of a nation-building exercise. It is not surprising then that the mandate of these services was to ensure the respect of national norms and regulations, which were supposed to contribute to the creation of a unified and standardized system. Supervisors had to undertake three sets of tasks in this regard: control (which is evident from their original name in many countries ­ "inspection"), support and liaison. This last role implies both vertical (informing schools of ministry policies and rules and informing the ministry of the needs and realities in schools) and horizontal interactions (networking between schools). It was believed that through these tasks, supervision would lead to improved school quality. The literature on school supervision has shown that until the 1980s there has been little fundamental change in its role or its organization (Carron and De Grauwe, 1997). Interestingly, even the decolonization process did not lead to a change in the school supervision system: the newly independent countries to a large extent adopted the existing supervision structures. This could be interpreted in two complementary ways: on the one hand, every state, whatever its character, feels the need to control what goes on in schools through some form of supervision; on the other hand, the decolonized state has never been very different from the colonized one, the main difference being that the "owners" of the state apparatus changed from an external to an internal elite. Some research on European countries has shown a similar consistency over time in the role and structure of supervision (Hopes, 1992). This should not be understood as a claim that not a single attempt at reform occurred. One recurring reform for instance aimed at emphasizing the support role of supervisors, which was reflected in many cases in a change of name, from "inspector" to "supervisor", "pedagogical advisor" or "resource person". However, in many cases not much more that a name changed, and such reforms did not touch the fundamental characteristics of the supervision system. The classical model 3
This supervision model which continues to exist in a great number of countries and has successfully resisted many reform attempts, can be called the "classical" model. It came about as a result of the adaptation of the supervision service to the expansion of the education system in line with the deconcentration of the administration that accompanied it. Supervision retains the role it was first assigned: that is, to control and provide support in pedagogical and administrative areas. In addition, coverage is supposed to be global: each school and teacher has a right ­ or could be submitted ­ to supervision.
In order to undertake this ambitious mission, supervisors find themselves in all the echelons of administration: at district level, where, in general, they exercise control over primary schools and provide support to teachers; at regional level, where they have the same tasks but in secondary schools; and at central level, where their role might include an evaluation of the evolution of the education system, as the General Inspectors in France do or a used to be the role of the Standards Control Unit in Zimbabwe.
Table 1: Central level Regional level District level School level
The structure of the classical supervision model
Responsible for the elaboration of supervision
supervision service policies, global planning, training and system
Responsible for supervision in secondary
supervision office schools, control of the development of
education in the region
Responsible for supervision in primary schools,
control of education development at district
Advisors and Advise primary and secondary school teachers
resource centres
Principal or head Informal supervision of teachers
This model can be called `classical' as the essence of the supervision exercise has little changed since its creation. Even though there have been some reforms in response to some demands of teachers ­ for example the creation of pedagogical advisors or greater transparency by announcing visits and systematic debriefing sessions ­ these innovative elements have not profoundly modified the service.
This model was implanted in most developing countries, particularly in the previous British and French colonies in Africa as well as Asia (Carron et al, 1998; De Grauwe, 2001). In its pure form, this model places a strong emphasis on the external supervision service, which is the most important monitoring tool. The internal evaluation of the school is weak and exam results can be used to inform the supervision process, but play no further role in controlling schools. This reflects a strong trust in the capacity of the State to control schools effectively. Countries such as Cuba and Viet Nam have kept this model without much change, partly because of the easy integration of its key principle ­ careful control by the State over the implementation of central rules ­ into the existing structures.
Research on school supervision in countries with this model has generally shown a great level of dissatisfaction with its impact on schools. There are a range of reasons for this; they are mainly related to the wide discrepancy between the very demanding mandate of school supervision and the scarce resources attributed to it, and to its complex bureaucratic structure. In fact this model was originally developed in countries where the services of the state were effective and well
financed and was then in some cases implanted into an almost totally different environment: a weak state without resources. The model works best in countries that have a competent public service, with civil servants that are rather well paid. Among developing countries, one can think of Botswana, Malaysia or Viet Nam.
These countries and others where this "classical" model works to the satisfaction of teachers and decision-makers are a rare exception and even among OECD countries, where resource constraints are less of an issue, dissatisfaction has been strong. Against this background, in the late 1980, a number of Anglo-Saxon countries undertook a radical reform of their school management system, with equally radical implications for the supervision service. These reforms were an expression of two closely related arguments. Firstly, the "classical" supervision service is ineffective because it tends to focus on two almost contradictory tasks (control and support) and because it does not sufficiently separate those who provide services (i.e. the educational administration) from those who evaluate them (the supervisors form indeed part of the administration). Secondly, these criticisms are addressed not only at the supervision service, but at the state as a whole: its action is ineffective because it tends to try to do too many things and because the public administration works for its own interests rather than those of its customers.
The central control model
The reforms, which were inspired by the above-mentioned criticism, have led to the development of what we will call the `central control' model. This model is based on the following convictions: · supervision should concentrate on one task ­ control. It is harmful to ask supervisors to combine support and control as the conflicting roles that this entails renders ineffective their interventions in the two domains; · the heavy bureaucracy that characterizes the classical model is not only expensive, it also prevents it from functioning effectively: there are too many small offices and the different levels lengthen the time between the supervision visit and follow-up to its recommendations; · external supervision cannot on its own lead to school improvement. This is the responsibility of the actors at school level (the principal, the teachers, the board, the parent association). But school inspection can be an incentive to start internal school reform, by informing the school and the public of the school's progress and weaknesses.
In this model, the role of the supervision service is therefore fairly simple: to inspect each school from time to time and to publish a public report. Such an inspection and its report examine all the aspects of the school's functioning and could be considered an `audit'. The structure of this model, which is presented in table 2, reflects its role: strong central control and few, if any supervisory actors at lower levels, while support is made available through private providers.
Table 2: The structure of the central control model
Central level Central inspection In charge of full inspection of all schools
body (autonomous) every 3, 4 or 5 years and informing the
Regional level
No specific officers
District level
No specific officers
School board
Supervision of school management
Regular supervision of teachers; decides on the need to purchase advice from private providers
Private service
Private providers
Offer advice to schools and teachers upon their request
This model exists in its most pure form in several Anglo-Saxon countries and in particular in England & Wales and in New Zealand. In both countries, as we mentioned before, the construction of a new system was intrinsically linked to a more global reform of the public service and the management of the education system. The context of this reform was an economic crisis and strong criticism of the public service ­ the public education system in particular. The system of inspection was also criticized: it was accused of being characterized by a heavy inefficient bureaucracy, a derisory impact on school improvement and a body of conservative and individualist inspectors. These criticisms were also fed by more ideological arguments: the ineffectiveness of the supervision system was seen as proof of the ineffectiveness of the State, in particular when it comes to evaluation of its own agents. Evaluation can best be done by an actor outside of the educational administration, which is why autonomous bodies were created. In the same vein, the inclusion of the public in the evaluation process will break internal complacency. Indeed, in the UK, "lay" inspectors, namely members of the public without any training in school supervision, form part of the inspection teams
These criticisms brought about a profound restructuring of inspection. In New Zealand, a very classical structure was replaced with an independent unit, the `Education Review Office', while local and regional offices were abolished. This Office has a mandate to inform the Ministry and the public of the effectiveness of the system and all its schools. Each school is inspected every three years. During these visits, the review officers do not offer formal support. Schools are expected to use their own budgets to buy support (for example training courses) offered by universities and other training institutions. The report is a public document and contains a summary that is specifically addressed to the local community. The Office for Standards in Education (OFSTED) in England and Wales follows to a large extent the same approach.
This model relies on inspection visits and reports as its main monitoring tool. However, because such supervision takes place only every few years, intermediate information is needed to monitor the schools' performances. This is why in the same countries the exam and assessment test system has generally been strengthened, and the schools' results on these exams and tests are used as information for the schools and the public. The publication of results in league tables has become probably the best-known and most controversial form of intervention in the monitoring system in a country such as the UK. The use of exam and test results to measure ­ at central level ­ the performance of schools and to hold them (and the local authorities who manage them) accountable, is also a key characteristic of recent monitoring reforms in the USA, where the No child left behind Act of 2001 has strengthened in this manner central control over what used to be very autonomous local school boards (Carnoy and Loeb, 2001; Stecher and Kerby, 2004).
Several countries have adapted certain elements of this model, particularly the institutional audit carried out by a specific corps of inspectors (there were attempts to introduce such audits in the beginning of this century in several countries, including Guyana, Malaysia and Zambia). These countries have nevertheless kept a classical supervision process, which in principle concentrates more on support than control. The objective of the audit is to reinforce the evaluation of schools and give it a formal structure and character. This allows for a more intensive use of the reports of these audits which remain, however, confidential, in contrast to the situation in for example England.
The close-to-school support model
Rather few countries have gone down the radical road of reform described above. As we mentioned before, most have kept the classical model in place. A few though have adopted reforms which have deeply changed supervision, but in a completely different way from the central control model. Their reforms are also based on a criticism of the classical model, but they draw very different conclusions.
They are based on the following reasoning: the main weakness of the classical model (and of the central control model) is to consider all schools as rather similar units. The supervision system can therefore treat all schools as equals and use the same strategies towards all. But schools have very different characteristics: their environment, pupils, teachers, parents, resources and so on are all specific to each school. And, because they have different characteristics, they also have different needs. While the best performing schools can function without any external support or supervision because of their internal strengths, this is not at all the case for the weaker schools. The supervision system should take those diverse needs into account. What those `weaker' schools need is not control alone but consistent pedagogical support and therefore regular visits by support-oriented supervisors.
In this model, the core role of the supervision service is to assist the weakest schools by offering them advice and guidance on how to improve. With such a purpose in mind, each school will need to be treated differently and supervision will have to adapt itself to the needs of each school. The drawback of the `classical' model is precisely that by trying to cover all schools without distinction, it fails to give due attention to those schools most in need of its intervention.
These points have implications for the supervisory structure. To enable supervisors to make regular visits, most are based as close to the schools as possible, while central and provincial officers no longer visit schools, but are in charge respectively of policy-formulation and training. To avoid supervisors spending too much time on administration, a specific cadre of administrative controllers may be created. And to ensure that they focus on the schools most in need of their support, a database identifies a fairly limited number of schools with which each supervisor has to work. The following structure is thus developed.
Table 3: Central level Regional level District level School
The structure of the close-to-school support model
Central supervision Small team in charge of development of
supervision policies
Regional supervision Small team in charge of training
supervisory officers
District Supervision In charge of offering intensive and
development-oriented supervision to those
schools most in need
In charge of controlling in particular the
finances of all schools
Informal supervision of teachers
The purest example of this model was developed in Chile, after the end of the Pinochet regime, when a new democratic government came in power. While under Pinochet the overall performance of the education system had improved, this had been accompanied by an increase in disparities. By giving attention to equity, the democratic government tried to strengthen its legitimacy. Its equity-focussed policies were not limited to supervision, but guided social policies as a whole, though some remnants of Pinochet's ideology could not be abolished, for instance
parental free choice of school, which worked to the detriment of equity (Baeza and Fuentes, 2003; Avalos, 2004). Supervision visits, in this model, are an important monitoring tool. However, the character of these visits is very different from the previous models: the supervisor visits a few carefully selected schools and tries to develop a close relationship with them. The visit includes classroom observation, workshops with the school staff, discussions with all teachers and with the school community. The aim is to develop together projects and plans to improve teaching and school functioning. While few countries have adopted a system such as Chile's, several projects have integrated the key principles of this model, namely a flexible development-oriented support to the most disadvantaged schools. This is the case for instance of programs in Sri Lanka (Perera, 1998), Bangladesh (Govinda and Tapan, 1999) or Kenya (AKU, 2000). In all these examples, there is a close linkage between the external supervision and the school's self-evaluation. The supervisor, when in school, works with the school's staff on identifying its strengths and weaknesses and on developing a school improvement plan. Supervision becomes thus a stage in the process of school self-evaluation and improvement, while in the preceding model the school's selfevaluation is a phase in the external inspection process. In other words, in this model external supervision helps the school undertake its own evaluation, while in the central control model selfevaluation helps the external inspectors to carry out their inspection. Exams play an important role, namely to allow the Ministry and the supervision service to know which schools to focus on and to monitor the reduction of disparities. Their role in monitoring schools is thus very different from in the previous model, where exam results are public information and parents use them to choose a school. The school-site supervision model While nearly all countries have some form of external school supervision, a few have either never had such a service or have more recently abolished it. As such, the non-existence of external supervision can be described as a fourth model of school supervision. We will see indeed that the absence of external supervision does not equal absence of all forms of supervision. This fourth model has a different history from the two previous ones: it was not systematically developed in reaction to the inefficiencies of the `classical' model. It is to some extent typical of countries with the following characteristics: great homogeneity, a society with few disparities, well-motivated teachers, public trust in their professionalism and strong parental interest in education. In such an environment, the teachers and the local community might appear the best monitors of the quality and the functioning of the school. These two groups are sufficiently close to the classroom to have a direct impact on the teaching process. The conviction exists, moreover, that the teaching staff have the skills and professional conscience to participate in selfand in peer-evaluation without being supervised from outside, and that the local community is willing and competent to exercise some control over the school. Moreover, because of the low level of economic disparities and because of the cultural and social homogeneity, there is little need for strong central intervention, either to address those disparities or to ensure the respect of national norms, including the curriculum. In other words, there is no need for a formal supervision service organized by the Ministry of Education. Countries where this model exists are also characterized by a fairly high level of school autonomy. The Scandinavian countries but also some States in the USA and Canada and some cantons in Switzerland are among these (see for instance Lacey, 1999, on Ontario; Duru-Bellat and Meuret, 2001, p.183 on Rhode Island and Favre, 2001 and Strittmatter, 2001, on Switzerland). At the local level, different scenarios can exist. The self-evaluation can be very 8
informal, without much structure or organization, relying on the individual initiative of the teachers; or it can be the responsibility of a specific structure such as a school governing board, which can be in charge of one or a few schools. While there is no external supervision, there are central-level tools to monitor the schools, such as examination and test results and indicator systems.
The following table shows the structure of this model, where all supervisory actors are based at the school-site, at local level or in the school.
Table 4: The structure of the school-site supervision model
Central level Regional level District level Local level School
supervision officers
supervision officers
supervision officers
School board or
Headteacher and
senior staff
All staff
No external school inspection as such, reliance on indicator systems, examination and test results In charge of supervision of the management of the school: the role of the headteacher Regular supervision of teachers; decide on the need to ask advice from teacher training officers Involved in school self-evaluation and development of school improvement plans
Finland is an example of this model (see for instance Webb and Vulliamy, 1998; Kivirauma et al, 2003; Webb et al, 2004) .Its external supervision service was abolished in 1991. Decision-makers felt that the benefits from supervision services were minimal and that, in view of the high level of training and professionalism of teachers, quality control could be entirely entrusted to them. In the same vein, the strict national curriculum was replaced in 1994 by a much lighter framework. Schools were encouraged to undertake their own evaluation, although no national strategy was developed imposing upon schools to undertake self-evaluation. Several schools nonetheless took such an initiative, pushed into doing so by the municipality. Allowing schools so much autonomy in their evaluation however does not mean that the central government is not preoccupied with the quality and functioning of schools. Their preoccupation is expressed in at least two ways: · the Ministry organizes optional achievement tests, develops national performance indicators and proposes evaluation procedures that the municipal level can employ. A `National Board of Education' has been set up that, among other things, evaluates the education system through for example examining the operations of a sample of educational institutions; and · the abolition of the inspection service and of the national curriculum was counterbalanced by the development of a framework, with norms and indicators that allow the Ministry to compare between schools.
In the other Scandinavian countries, we see a similar evolution, with an earlier movement away from central control towards local-level regulation being tempered more recently by the reintroduction of some form of central regulation (Eurydice, 2004, p.31-32). It is not surprising that such a model is not present in most developing countries precisely because of the absence of the characteristics which explain its success in places such as Finland, namely a strong teaching corps, closely involved and well educated parents and few disparities.
It will have become clear that, in the absence of external supervision, the role of the other two monitoring tools, exams and assessment and self-evaluation, have grown in importance. The
absence of external supervision is not so much a result of doubts about the effectiveness of State control as a reflection of the relative strength of the other actors who can exercise control, i.e. the teaching profession and the local community. Neither does absence of external supervision imply a naive belief in the intrinsic value of teacher autonomy. It could be argued that, where these other tools and actors work properly, teachers might have less autonomy in their classroom than in a system which relies mainly on an external supervision system that is not functioning efficiently. Models and disparities The relationship between these different models and the level of disparities between schools is complex. Three questions can help us clarify this relationship: firstly, what attention do these models give to the issue of disparities: do they recognize its existence or do they consider it a problem of little importance? Secondly, what do these models consider as the causes for such disparities? Thirdly, what impact do their interventions have on the level of disparities? The first and fourth model do not pay much attention to this issue, while the second and third do but offer almost contrasting responses. The first model ­ the classical supervision ­ gives very little attention to the issue of disparities. This finds its explanation to some extent in the origins of this model: it was developed in the context of an elitist system and its original aim was to ensure the development of a homogeneous system, as part of a nation-building exercise. Supervision was very much a tool of standardization, preoccupied with enforcing respect of national rules and regulations. This was linked to the conviction that the respect of these rules, in the form of a national curriculum and nation-wide norms on e.g. teacher qualifications or pedagogical methods, was the best strategy to help each school achieve good results. Any movement away from these norms, even if intended to make school more locally relevant, was considered an abnormality. This reasoning can still be valid under two conditions. Firstly, when a country is relatively homogeneous and with little disparities, supervision as a standardization tool can have a positive impact and may not worsen disparities. In such a scenario, applying the same framework and norms throughout the country may indeed make sense. A second condition relates to the State having sufficient resources to ensure that its supervision reaches out to all schools on a fairly regular basis, as this is needed to guarantee the respect of these standard rules and regulations. While these two conditions may to some extent still be present in a few countries, they are absent in most places which work with this classical model, even in some of the most developed countries where disparities have increased and resources are insufficient for regular coverage of all schools. Several responses are then possible. In France for instance, the classical supervision services have changed little, but special programmes have been created to give additional resources and support to the most disadvantaged schools, the Zones d'йducation prioritaire. These compensatory programmes, which think along the same lines as the third model described above, tend however to lead to a stigmatization of the schools which belong to these priority zones and their record is patchy. In developing countries, the key constraint is that the mandate of supervisors, namely to supervise and support all schools, outweighs by far the resources. A rather unpromising response, which many supervision services nevertheless adopt, is to ask continuously for more resources, which however never arrive. Another option is to accept the limited level of resources and to restrict the mandate, i.e. to cover a limited number of schools. If disparities between schools would be of great concern (and in many developing countries they are of such a high level that they should be), then supervisors probably should focus their attention on the schools which 10
need their intervention most. Unfortunately many supervision services in developing countries have made a different choice, namely to focus on the schools which are within easy reach, such as those close to their office or to the main road. The result is that more remote schools tend to get less supervision. In Zimbabwe for instance at the end of the 1990s, while a teacher was on average visited every two and a half years, those in rural areas had to wait four years (Moyo et al, 2001, p.269). There are at least four reasons to explain this scenario. Firstly, because these schools are easier to reach, they are less expensive to cover. Secondly, because supervisors are to some extent evaluated on the number of schools and teachers supervised and the number of reports, it makes more sense to visit many schools close by even if they do not need such a visit, rather than a smaller number of isolated schools, even if they are more in need. In other words, institutional incentives are absent for these officers to change their traditional practices. Thirdly, the administrative structure is such that all districts or regions within the country are assigned a similar number of administrative or supervisory staff, without taking into account the needs and characteristics of schools for which they are responsible. Finally, few supervisory offices have developed within their own district the necessary data and indicators allowing them to carefully select schools most in need. Because of this complex set of reasons, in many developing countries the standardized supervision service is probably reinforcing disparities, because it is reaching out more to better-off schools than to schools facing difficulties. To summarize, the classical supervision model is in most developing countries no longer suitable to the existing education system. Not only does it fail to fulfil its mandate ­ comprehensive administrative control ­ but its mandate itself has become outdated: while some form of control is undoubtedly needed, the focus must shift to pedagogical improvement. It could be hoped then that more support-oriented strategies, introduced precisely to make up for the over-emphasis on administrative control, would be more successful in addressing disparities by concentrating their efforts on the more disadvantaged schools. However, services which specialize in pedagogical support suffer from a somewhat comparable weakness. In many case, their advice is also benefiting mainly the schools closest to where these support services are located. Research undertaken on the role of resource centres for instance in India, Nepal, Kenya and Zambia (Khaniya, 1997; Knamiller, 1999) shows that they are generally not able to reach out to a large number of schools, and even where they succeed in doing so, they offer advice which is of little relevance to the situation of schools whose resources and context are too far away from the standard one that these services know and cater for. The interventions by these school monitoring services are reflective of the State's intervention as a whole: because the State is incapable to fulfil its mandate, authorities tend to focus on those groups whose support is important to their survival. The politically less vociferous groups are to some extent abandoned and will at times, with their own scarce resources and with the help of non-governmental organizations, set up their own services. The fourth model ­ the school-site supervision model ­ pays little attention to disparities, mainly because, as we saw above, this model was developed in countries where such social and educational disparities are weak. The absence of these disparities is not a simple coincidence; it is the result of socio-economic policies aimed at reducing disparities. These socio-Democratic traditions are strongly anchored in the psyche of these countries. This in itself offers an important insight: reducing disparities is probably not the key role of education, which cannot be asked to repair what socio-economic policies have failed to solve or may even have contributed to. Educational policies evidently can have an impact on disparities, but as long as they are not an integral part of an overall framework with the common objective to reduce such disparities, they 11
will never succeed in a sustainable diminution of these disparities and will always be fighting a rearguard battle. This also brings up the point how far this "peer-supervision" model can be transplanted from one national context to another. It relies on a distant evaluation by the centre, a strong trust in the teaching corps and the involvement of a capable and interested community. Where disparities are important, such a model may not be able to address them because there is too little external intervention in schools. Even in Finland, one of the economically most egalitarian societies, differences in the level of school evaluation are developing as a result of the actions by municipalities. Indeed, some big urban municipalities have set up their own school evaluation units, believing that these will contribute to higher quality. In more remote rural regions, school principals and teachers are not at all supervised by any external actor, mainly because municipalities do not have the necessary financial or Human Resources to organize supervision (De Grauwe, 2006). In the long run, this could lead to increased disparities for which the actual system is not prepared. The central control model does pay attention to the issue of disparities. It puts the blame for differences in results between schools squarely on the school's internal functioning. It is based on the conviction that as long as schools follow a certain set of principles, based on a rather simplified interpretation of school effectiveness research, they will perform with success. The supervision service imposes these principles upon all schools and therefore uses the same approach systematically. The school's socio-economic context is disregarded. Studies on the UK (Smith, 2000) and New Zealand (Fiske and Ladd, 2001) which implemented fairly pure examples of this model, emphasize this lack of attention to the school's context, while at the same time pointing at the possible linkage. Smith's study demonstrates a strong association between the socio-economic context and the evaluation made by OFSTED: among schools based in the highest socio-economic areas, 91 % received a "favourable" evaluation and only 3% a "not favourable" one; among schools in the lowest socio-economic areas, only 2 % received a "favourable" evaluation and 89 % a "not favourable" one. When OFSTED made efforts to take more into account the school's context in its inspection visit, there was resistance from among political authorities who felt that such a move could be interpreted as a lowering of expectations and a loss of pressure on schools to perform. More preoccupying however than this neglect of the context, is the fact that the supervision strategy followed by the central control model tends to enlarge disparities between schools. A quantitative analysis undertaken by Shaw et al (2003) indicates that in only three types of schools (girls only schools, selective schools and grant-maintained schools, all of which are schools catering for a certain elite public) inspection was associated with a light improvement of exam results. Elsewhere, there is no impact or results seem to decrease. A similar conclusion was arrived at by Cullingford and Daniels (1999) on a less representative sample. Chapman (2002) went in some more detail and examined what changes schools in difficult circumstances undertake following an inspection visit. These changes have little to do with what takes place in the classroom but mainly concern administrative reporting or management procedures. Chapman draws the conclusion that a monitoring approach such as OFSTED's is better at influencing management practices than Classroom Interactions. Unfortunately, what such weak schools need above all is a change in classroom behaviour. Research in New Zealand (Fiske and Ladd, 2001) has also led to the conclusion that the central control model tends to lead to increasing rather than reducing disparities. There are at least two reasons for this. Firstly, schools and teachers receive too little support. Many teachers make the point that recommendations for improvement are of little help if they 12
are not accompanied by advice and guidance on how to implement these recommendations. The best performing schools evidently suffer much less from this lack of support than the weaker schools. Schools in a situation of crisis feel de-motivated by a process which highlights their weaknesses (and turns them into public knowledge) without offering any solutions (Fiske and Ladd, 2001; Case et al, 2000; Shaw et al., 2003; Chapman, 2002; Cullingford and Daniels, 1999). Moreover, the competition between schools which this model promotes, has led to a weakening of school networks, to the detriment of the weakest schools, as Harold (1999, p.11-12) demonstrates for New Zealand. Secondly, this central control model puts a heavy emphasis on the responsibility of schools and teachers. The school principals in particular suffer under this stress, when they are made responsible for the school's performance, even if certain factors explaining this performance are beyond their control. Because of the emphasis on accountability, principals also tend to spend much of their time on administrative and managerial tasks, cutting in their time available for supporting their teachers. Again, this is more of a concern in weaker schools. It is not surprising that in South-Africa, where attempts were made to introduce a similar model, the only schools who have easily accepted to undergo a "whole school review" are those who recruit from among the richest parents and who have the best results ­ the previously "white" schools. The positive evaluation reports they get, are helpful in their public relations campaigns. A further negative effect of the central control model is that in a number of cases it has delegitimized the internal school evaluation process. Indeed, teachers who have gone through a school audit, which internal evaluation helped to feed and which led to a critical public report, feel to some extent "cheated". They become somewhat suspicious about any evaluation process, including those led by internal actors and aimed at school improvement (Webb and Vullimay, 1998, p.548). It also makes collaboration within the school more intricate. This is deplorable because all schools, including those with the poorest results, need such an internally led process in order to improve. While an external encouragement is in many cases necessary, it should not pervert the objectives of the internal process. It is because of these various criticisms that OFSTED itself has taken a number of steps which have moved it away from this pure model: there is more space foreseen during the supervision visits for discussion with teachers; reports contain some helpful advice; the school's environment receives some attention. Behind this central control model, we discover a model of a State, the main preoccupation of which is not to guarantee equity, but to ensure that public services respond effectively to the demand of the individual consumers. This has different implications. market mechanisms are promoted as they are believed firmly to serve both individual and common interests. For this market to function well, the State needs to provide the consumers with the necessary information. The game of choice and competition will lead to an overall improvement of schools, and to the disappearance of those schools who fail to improve. This demands that these services are subject to public accountability and that their functioning becomes more transparent. The close to school support model ­ our third model ­ is very much interested in the question of disparities. It offers specific attention to the context of schools and recognizes that schools function in different socio-economic environments and that these may have an impact on their performance. Supervision therefore needs to adapt itself to the needs and the characteristics of each individual school. This supervision model is illustrative of an overall concern with equity. This concern is based on the conviction that the State has the mandate, if not the obligation, to intervene in favour of the least advantaged and the weakest actors in society, including through policies of positive discrimination. This equity-focused policy also offers the State a source of legitimacy. The model is also popular with non-governmental organizations, who work precisely within a State which does not pay attention to disparities. These NGO's can leave administrative 13
control to the official authorities and dedicate their time and efforts to the most disadvantaged sections of society (or, in the education system, the most disadvantaged schools). The success of this model, when it is the initiative of a government, depends therefore in part on the characteristics of this government. Setting up a supervision service, which has as an objective to offer a collegial-type support as part of a compensatory programme, has little chance of succeeding if it is the initiative of a government which for the remainder shows scant interest in equity and whose other interventions are focussed on control. The relative failure of the resource persons in Nepal (Khaniya, 1997) or of the Groupes d'Appui Pйdagogique in Morocco (Yeklef and Tazi, 2004) are testimony to this. However, the State does not only need to have the right intentions and policies, it also needs to be able to put these into practice. The South-African government for instance (or, to be more correct its federal government and those of several provinces) has a series of compensatory policies (such as giving additional financial resources to the schools in the poorest areas) but has not been able to transform the supervision service, which is indeed a more complex exercise, encountering resistance both from practising supervisors and from teachers. Unfortunately, those schools and teachers who in principle could benefit most from a more support-oriented supervision service, have, because of the memories of the apartheid regime and its inspection system, resisted almost any re-introduction of supervision into their schools. While the close-to-school support model seems the most promising one to address disparities, its introduction encounters several difficulties, even by governments who have an overall equityfocussed policy and have the necessary resources. We commented earlier on on some of these challenges. It is relatively straightforward to adapt the rhythm of supervision to the results of the schools, with less supervision for well performing schools and more for those with poorer results. It is much more difficult to change the character of the visits themselves, and to demand that each visit changes in function of the needs of each particular school. Several authors have commented that in Chile visits by the same supervisor differ very little from one school to another. A related difficulty, and probably the most complex one, is linked to the enduring attitudes of supervisors, who are used to giving directions and find it difficult to develop a genuine collegial relationship with teachers, especially with those who are not doing a very good job. Few of the supervisors are recruited from among the teachers who have worked in the most remote or disadvantaged schools and may not easily understand the specific constraints of such an environment. Behind this model lie finally two internal contradictions, which may work to the disadvantage of the weakest schools. The first one is well known: while weaker schools in this model benefit from more support, they are in this way assigned to a group of less successful schools, and this can work against them, especially if there is a parental right to choose a school. Secondly, while it is correct that disadvantaged schools benefit from more support under this model, it is equally true that the stronger schools, who regularly recruit from among the higher social classes, gain the advantage of greater pedagogical and managerial autonomy. This is partly the result of the political strength of their public. To some extent, the fact that the successful schools can operate without much intervention by governmental representatives, is an admission by the government that it may not have the capacity to regulate the whole school system. This could become a problem under a scenario whereby the resources of the better off parents, who support their schools through school fees and the like, outweigh the resources of government. This is indeed the case in South-Africa, which helps to explain why disparities between schools have not decreased swiftly. While this is not an ideal scenario, it is nevertheless preferable over one whereby the government distributes its scarce resources to those who have already. Conclusion: the complexity of reform 14
Reforming school supervision is in all cases an intricate exercise, and becomes even more complex when the objectives of such reforms are to develop a supervision system which aims at reducing disparities. This conclusion cannot discuss in detail all the challenges to the process of policy reform. We will highlight what are two fundamental ones. Firstly, cultures of organizations can be extremely strong, especially when these organizations are made up of fairly well qualified professionals and when these professionals have gained access to this organization through having shown respect for this culture. This is very much the case with supervisors: they have generally been chosen from among the teachers who show the greatest respect for rules and regulations. Their profession consists mainly of imposing respect for these same rules. This leads to a strong culture, resistant to reform imposed from outside. As the Chilean example, on which we commented earlier, shows, reform of such a culture is possible, but demands a set of integrated strategies and is facilitated by radical political change. This brings us to the second fundamental challenge, which resides in the political economy of reform. It is well known that reforms which aim at lessening disparities through some form of AFFIRMATIVE ACTION, positive discrimination or compensatory programmes encounter at least two forms of resistance. Firstly, such programmes may seem unfair when one disregards the wider social and economic discriminations in society, built upon a long history. There is a need for public sensitization. Secondly, and maybe more seriously, the political decision-makers in society seldom belong to the groups who suffer from discrimination. If anything, they form part of the elite, who benefits from the inequalities in society. Equity-focussed policies are therefore easiest to develop in moments of deep political reform. Anton De Grauwe Bibliography AVALOS, Beatrice. Teacher regulatory forces and accountability policies in Chile: from public servants to accountable professionals. Research papers in education, 2004, 19(1), p.67-85. BAEZA, Juan, FUENTES, Ricardo B. Antecedentes y fundamentos de las polнticas de gestiуn y administraciуn en el sistema educativo chileno. 1980 ­ 2003. Santiago, Ministerio de Educaciуn, 2003. 64 p. BARROSO, Joao, AFONSO, Natйrcio, BAJOMI, Ivan et al. Systиmes йducatifs, modes de rйgulation et d'йvaluation scolaires et politiques de lutte contre les inйgalitйs en Angleterre, Belgique, France, Hongrie et au Portugal. Synthиse des йtudes de cas nationales, 2002. 73 p. (Reguleducnetwork ­ deliverable 3). BERMAN, Edward H. The Political Economy of Educational Reform in Australia, England and Wales, and the United States. In ARNOVE, Robert and TORRES, Carlos Alberto, eds. Comparative Education: The Dialectic of the Global and the Local, Lanham MD: Rowman and Littlefield Publishers, 1999, p. 257-282. BRAY, Mark. School clusters in the Third World : making them work. Paris : UNESCO, 1987. 147 p. BURNS, John. `Improvement through inspection'?: an investigation of teacher's perceptions of OFSTED as a vehicle for improvement. Oakhill: Trentham Books, 2000. 39 p. CARLSON, Beverley. Achieving educational quality : what schools teach us. Learning from Chile's P900 primary schools. Santiago, ECLAC, 2000. 72 p. CARNOY, Martin, LOEB, Susanna. Does external accountability affect student outcomes ? A cross-state analysis. Educational Evaluation and Policy Analysis, 2002, 24(4), p. 305-331. 15
CARRON, Gabriel, DE GRAUWE, Anton. Current issues in supervision: a literature review. Paris, IIPE, 1997. 72 p. CARRON, Gabriel, DE GRAUWE, Anton, GOVINDA, Rangachar. Supervision and support services in Asia. Paris, IIPE, 1998. 2 vol. CASE, Peter, CASE, Susan, CATLING, Simon. Please Show You're Working: a critical assessment of the impact of OFSTED inspection on primary teachers. British Journal of Sociology of Education, 2000, 21(4), p. 605 ­ 621. CHAPMAN, Christopher. OFSTED and school improvement: teacher's perceptions of the inspection process in schools facing challenging circumstances. School leadership & management, 2002, 22(3), p. 257-272. CHAPMAN, Christopher. Improving schools through external intervention. London: Continuum, 2005. x ­ 176 p. CULLINGFORD, Cedric, DANIELSandra. Effects of OFSTED inspections on school performance. In CULLINGFORD, Cedric ed. An inspector calls. London: Kogan Page, 1999. p.59-69. CUSACK, B.O. An end to school inspection: The New Zealand experience. Management in Education, 1992, 6(2), p.6-8. DE GRAUWE, Anton. L'Etat et l'inspection scolaire. Analyse des relations et modиles d'action. Doctoral thesis. Paris: Institut d'йtudes politiques, 2006. DE GRAUWE, Anton. Improving quality through school-based management: learning from international experiences. International review of education, 2005, 51(4), p. 269-287. DE GRAUWE, Anton. School supervision in four African countries. Volume 1: Challenges and reforms. Paris : IIPE, 2001. 154 p. DEMAILLY, Lise ; DEUBEL, Philippe ; GADREY, Nicole et al. Evaluer les йtablissements scolaires : enjeux, expйriences, dйbats. Paris : L'Harmattan, 1998. 303 p. DURU-BELLAT, Marie, MEURET, Denis. Nouvelles formes de rйgulation dans les systиmes йducatifs йtrangers : autonomie et choix des йtablissements. Revue Franзaise de Pйdagogie, 2001, 135, p.173-221. DUTERCQ, Yves. Administration de l'йducation : nouveau contexte, nouvelles perspectives. Revue franзaise de pйdagogie, 2000, 130, p.143-171. EURYDICE. L'йvaluation des йtablissements d'enseignement obligatoire en Europe. Bruxelles : Commission Europйenne, 2004. 156 p. FAVRE, Bernard. Analyse de fonctionnement, йvaluation et auto-йvaluation des йcoles : le cas des йcoles primaires genevoises. Perspectives, 2001, 31(4), p.614-631. FERGUSSON, Val. Supervision for the selfmanaging school: the New Zealand experience. Paris: IIPE, 1998. xi ­ 58 p. FISKE, Edward, LADD, Helen. Les йcoles autonomes et le devoir de rendre des comptes en Nouvelle Zйlande. Perspectives, 2001, 31(4), p.633-650. GARFORTH, Laura. Supervision of primary schools in Bolgatanga district: constraints and ways forward. Document non-publiй. S.l. 2004. GIORDANO, Elizabeth. School clusters. Fundamentals of educational planning. Paris: IIEP, 2008. GOVINDA, Rangachar, TAPAN, Shahjahan. Quality education through schoolbased supervision and support: the case of GSS primary schools in Bangladesh. Paris: IIPE, 1999. 65 p. GRAY, John, WILCOX, Brian. In the aftermath of inspection: the nature and fate of inspection reports and recommendations. Research Papers in Education Policy and Practice, 1995, 10(1), p.1-18. GUMBI, Constantine, DLAMINI, Barnabas. Perception of head teachers regarding supervision of Swaziland school system. Boleswa Educational Research Journal, 1997,14, p.47-59. GURR, David. From supervision to quality assurance: the case of the State of Victoria (Australia). Paris: IIEP, 1999. 137 p. 16
HANLON, G. Professionalism as enterprise: service class politics and the redefinition of professionalism. Sociology, 1998, 32, p.43-63. HAROLD, Barbara. A decade of selfmanagement in New Zealand schools: what have we learnt? Paper presented at AARE ­ NZARE Conference, 1999. HOPES, Clive. The role of inspectorate and inspectors in the development and monitoring of school management and effectiveness. Paris: OECD, 1992. 27 p. HOPPERS, Wim. Teachers' centres and resource work in Southern African Education; An investigation into decentralization and educational reform. Unpublished paper presented at the NASEDEC 1996 Conference, Copenhagen, 17-19 October 1996. 12 p. JOHANNESSON, Ingolfur, LINDBLAD, Sverker, SIMOLA, Hannu. An inevitable progress? Educational restructuring in Finland, Iceland and Sweden at the turn of the milennium. Scandinavian Journal of Educational Research, 2002, 46(3), p. 325 ­ 339. KIVIRAUMA, Joel, RINNE, Risto, SEPPANEB, Piia. Neo-liberal education policy approaching the Finnish shoreline? Journal for Critical Education Policy Studies, 2003, 1(1). KHANIYA, Tirth Raj. Teacher support through resource centres : the Nepalese case. Paris: IIPE, 1997. xi ­ 65 p. KNAMILLER, Gary, ed. The effectiveness of teacher resource centre strategy. London, Department for international development, 1999, 257 p. KOGAN, Maurice. Education accountability. An analytic overview. London: Hutchinson, 1988. 183 p. LACEY, Veronica. Building a national framework for accountability in educational reform initiatives. Paper prepared for the workshop on Quality assurance and Teacher Professionalism, Brasilia, 29-30 September 1999. LEVIN, Ben. An epidemic of education policy: (what) can we learn from each other? Comparative education, 1998, 34(2), p.131. LOCKHEED, Marlaine, VERSPOOR, Adriaan. (1991) Improving primary education in developing countries. Oxford: Oxford University Press, World Bank. 429 p. LUGAZ, Candy, DE GRAUWE, Anton. Ecole et dйcentralisation. Expйriences et dйfis en Afrique francophone. Paris : UNESCO, 2006. 144 p. MACBEATH John. Schools must speak for themselves: The case for school selfevaluation. London: Routledge, 1999. xi ­ 165 p. MCINERNEY, Peter. Moving into dangerous territory? Educational leadership in a devolving education system. International Journal of Leadership in Education, 2003, 6(1), p. 57-72. NAIDOO, Jordan. Educational decentralization and school governance in South Africa: from policy to practice. Paris: IIEP, 2006. 163 p. NAVARRO, Luis, PEREZ, Sonia, ARANGUIZ, Gabriel et al. La supervisiуn tйcnicopedagуgica en Chile. Paris: IIPE, 2002. 105 p. NEWTH, Ash, WHALLEY, David. School inspectorates of the Kingdom of Lesotho. Review. Auckland: Multi Serve Education Trust, 2002a. 68 p. NIEMI, Hannele. Recreating values. In NIEMI, Hannele ed. Moving horizons in education : international transformations and challenges of democracy. Helsinki: University Department of Education, 1999. p. 211- 228. OSLER, Audrey. L'inspection des йtablissements et l'inйgalitй raciale. In : EVANNO, Jean-Noлl, dir. Le « New Labour » et l'йducation : la « troisiиme voie » mis а l'essai. Rennes : Presses Universitaires, 2001. p.103-113. PAIR, Claude. Forces et faiblesses de l'йvaluation du systиme йducatif en France. Paris : Haut Conseil de l'йvaluation de l'йcole, 2001. 59 p. PELLETIER, Guy. De l'inspection aux nouvelles formes de rйgulation du systиme scolaire du Quйbec. Administration et education, 2005, 108, p. 107-113. 17
PERERA, Wilfred. Changing schools from within: a management intervention for improving school functioning. Paris: IIPE, 1997. 76 p. POSCH, Peter. Educational innovations and implications for the teaching force: the case of Austria. Research papers in education, 2004, 19(1), p.87-104. RICHARDS, Colin. Towards respectful inspection ­ a critique of proposed arrangements for school inspection and self-evaluation. Education review, 2004, 18(1), p.42-48. SAUNDERS, Lesley. Who or what is school self-evaluation for? School effectiveness and school improvement, 1999, 10(4), p. 414-429. SHAW, I, NEWTON, Douglas, AITKIN, Murray et al. Do OFSTED inspections of secondary schools make a difference to GCSE results? British Educational Research Journal, 2003, 29(1), p.63-75. SIMOLA, Hannu, RINNE, Risto, KIVIRAUMA, Joel. Abdication of the Education State or just shifting responsibilities ? The appearance of a new system of reason in constructing educational governance and social exclusion/inclusion in Finland. Scandinavian Journal of Educational Research, 2002, 46(3), p.247- 264. SMITH, George. Research and inspection : HMI and OFSTED, 1981-1996 ­ a commentary. Oxford Review of Education, 2000, 26(3-4), p. 333-352. SOLAUX, Georges. Les Politiques de gestion des personnels enseignants dans les pays d'Afrique subsaharienne francophone. Dijon : IREDU,1997. 250 p. SOLOMON, Joseph. Trends in the evaluation of education systems: school (self) evaluation and decentralization. Athens: Hellenic Ministry of Education, 1997. 135 p. STECHER, Brian, KIRBY, Sheila, Nataraj. Organizational improvement and accountability. Lessons for education from other sectors. Santa Monica: Rand Corporation, 2004. 154 p. STRITTMATTER, Anton. L'auto-йvaluation dans les йtablissements scolaires et le rфle des autoritйs scolaires. In DEMAILLY, Lise йd. Evaluer les politiques йducatives : sens, enjeux, pratiques. Bruxelles : De Boeck Universitй, 2001. p. 111-129. THRUPP, Martin. Exploring the politics of blame: school inspection and its contestation in New Zealand and England. Comparative education, 1998, 34(2), p.195-209. UEMURA, Mitsue. Community participation in education: what do we know? Washington: HDNED, World Bank, 1999. 36 p. WANZARE, Zachariah. Rethinking school inspection in the Third World: the case of Kenya. Alberta: University of Alberta, 2002. [Consultй le 22 janvier 2003]. Disponible sur WALFORD, Geoffrey. Does the market ensure quality? Westminster studies in education, 2001, 24(1), p. 23- 33. WATSON, Jacqueline. OFSTED's spiritual dimension: an analytical audit of inspection reports. Cambridge Journal of Education, 2001, 31(2), p.205-219. WEBB, Rosemary, VULLIAMY, Graham, HДMДLДINEN, Seppo et al. A comparative analysis of primary teacher professionalism in England and Finland. Comparative education, 2004, 40(1), p.85-107. WEBB, Rosemary, VULLIAMY, Graham. External inspection or school self-evaluation? A comparative analysis of policy and practice in primary schools in England and Finland. British Educational Research Journal, 1998, 24(5), p. 539-557. WILCOX, Brian. Making school inspection visits more effective: the English experience. Paris: IIPE, 2000. 84p YEKHLEF, Amina, TAZI, Malak. Inspection scolaire et dйcentralisation au Maroc : les groupes rapprochйs de l'action pйdagogique. Paris : IIPE, 2005. 135 p. 18

A De Grauwe

File: school-supervision-a-tool-for-standardization-or-for-equity.pdf
Title: untitled
Author: A De Grauwe
Published: Tue Nov 4 17:40:29 2008
Pages: 19
File size: 0.15 Mb

À propos de, 4 pages, 0.08 Mb
Copyright © 2018