Governance of data and artificial intelligence

By Cecilia Alemany, Development Alternatives with Women for a New Era (DAWN) and Anita Gurumurthy, IT for Change (ITfC).

"People worry that computers will get too smart and take over the world, but the real problem is that they are too stupid and they’ve already taken the world." [fn] Domingos (2017), p. 286. [/fn]

“The Data Revolution” has been promoted as a vital tool to help to achieve the SDGs or, at least, to better measure progress. Having access to massive amounts of data is seen as helpful for countries to plan, design and implement development and public policies in general. This chapter highlights concerns about this revolution and suggests how to rethink global governance for the digital era.

---

The redefinition of the principles, norms and policies (software) and the structures and institutions (hardware) of sustainable development governance is closely related to our capacity to adopt new rules and adapt international structures to govern data and Artificial Intelligence (AI) and their impact on our lives and rights.

AI has been defined as

a sophisticated application of technology whereby a machine demonstrates human cognitive functions such as learning, analysis and problem solving, and a collection of advanced technologies that allows machines to sense, comprehend, act and learn. [fn] Compiled by Galloway/Swiatek (2018). [/fn]

The big data revolution and associated machine learning (ML) technologies that allow new modes of production in which digital intelligence is a factor are ushering in a paradigm change. As platform companies like Amazon reorganise the value chain using AI tools to orchestrate logistics, market exchange is radically redefined. AI-led reorganisation is also altering production logics and structures in all sectors (and of course how and where we work) and decision-making at all levels (from national governance to urban development, law enforcement, credit, and public education and health investments). Thus, the ways in digital intelligence, generated from social interactions data (of people and things in a networked data environment) to produce profit marks a shift in the foundational structures of society and economy, requiring a new governance model.

Although data and AI related laws and policies are nascent in most countries (with the exception of the European GDPR), [fn] GDPR entered into effect during the first half of 2018. See https://ec.europa.eu/commission/priorities/justice-and-fundamental-rights/data-protection/2018-refo…; see also Zimmermann (2019). [/fn] concerns around the inherent biases in AI and consequences for fundamental rights, including the right to equality and non-discrimination, are being widely flagged today by civil rights groups. Employees of big digital corporations have also raised their voices against the weaponization of cyberspace through a state-corporate nexus [fn] See https://www.nytimes.com/2018/04/04/technology/google-letter-ceo-pentagon-project.html. [/fn] .

What is vital is to recognise is that data and AI governance needs a more comprehensive approach that addresses the individual and structural underpinnings of equality and justice. The digital context presents leapfrogging possibilities for the fourth industrial revolution, and digital intelligence obtained through processing of data can provide developing countries the wherewithal for structural transformation and competitive advantage. Data regulation must hence encompass concerns that tackle the multivalency of data, recognizing the inalienability of data in relation to personal identity, but also cognizant of data’s enclosure as an economic resource. This means the imperative to manage and regulate cross-border data flows, with due attention to personal data protection through a wide array of national and global data policies – digital industrial policies, trade policies, [fn] UNCTAD (2019). [/fn] social policies and development policies.

The data marketplace

Among transnational corporations, platform corporations are the most powerful, fuelled as they are by the algorithms (mathematical instructions that process data) that run on our data. Today, the business of sharing, acquiring and monetizing data has spawned a global data marketplace where data brokers and global platform companies use data for at least three types of business functions: 1) to input into their own production and innovation processes; 2) to personalise and target marketing; and 3) to sell for use by any third company, politician, agency or anyone who can pay. In sum, transnational companies use our data as their data.

It is near-impossible in this scenario for citizens to navigate the complex terrain of ‘consent’, often recommended as a solution in the data marketplace, to make decisions about which data to share, with whom and for what. Consent frameworks are also rendered ineffective by the fact that the line between personal and non-personal data is blurred. Also, in the absence of data localization policies and the capacity to process data for real time digital intelligence, national and sub-national governments, especially in developing countries, lack the means to have access to the data generated within their territory for their own domestic innovation policies and programmes. As late entrants into the data game, most developing countries also lack robust, machine readable data sets. As a result, these countries may not yet be ready and well prepared to set up the digital and data infrastructure, including the public goods adequate to this new digital era.

It is not surprising therefore that as many as 88 developing countries are resisting the US proposal at the WTO for an e-commerce agenda and have concerns in terms of their unpreparedness to benefit from cross-border e-commerce as well as to engage in negotiations.

Implications for developing countries

In the Spotlight Report in 2018, IT for Change explained how traditional economic power asymmetries are being refined through platforms - emerging “digital ecosystems that provide a new architecture for the economy”, noting that “developing nations are the mining grounds for data, at worst, and the back offices or server farms for low-end data processing, at best.” This includes countries “that have distinguished themselves as tech hubs that often develop innovation products and services only to release intellectual control and economic dividends to the tech giants of the global North”. [fn] IT for Change (2018a). [/fn]

Structural inequalities between and within countries are being reproduced in this digital economy, as the global South risks “becoming an unregulated innovation playground for technology giants to experiment in, if adequate and comprehensive policy measures are not developed that can govern their operations.” [fn] IT for Change (2018b). [/fn] Developing country challenges include: “lack of coordination for innovation, lack of ability to mobilize domestic resources, inability to create linkages, low resilience of the domestic entrepreneurial sector, tax avoidance, and the failure to regulate competition”. [fn] Gehl Sampath (2019). [/fn]

Critical policy frontiers such as labour, consumer protection, privacy, e-commerce, foreign investments and other areas that directly impact the livelihood rights of citizens cannot be conceded to immediate short-term gains that big platforms often usher in.

AI and public policy decisions

Many public policy decisions that shape citizens’ everyday experience are found not in legislative norms but in software codes and AI made by scientists and innovators in private (and monopolistic) settings. [fn] UN Secretary-General (2018), p. 9. [/fn] Policy-makers are not yet seized of the risks of delegating public and private decision-making to AI and ML. All countries need to understand the impact of deep learning and intelligent prediction models in public policy design and response, in order to realize the potential benefits, as well as to mitigate the risks, of these intersections (See Box V.1). Good policy can ensure that this can be the beginning of a ‘golden age’ of social sciences, a coming together of contextual complexities and statistical interpretations at a new level, thanks to data and AI.

---

Box V.1: Data-based discrimination

There is growing evidence that machine learning technologies – based on existing data, search results and user experience – reproduce structural disadvantage through discriminatory results. [fn] See Purkayastah (2018). [/fn] An analysis of racial discrimination in Airbnb, using identical profiles with different names, found that those typically from the African American community had 16 percent lower opportunities to rent. A similar analysis of BlablaCar found that drivers with Muslim or Arab origin had a 20 percent lower demand than those with French names and received lower payments. A study on Google searches in the USA found that African American name searches produced advertising on detention reports but this did not occur when using typical white American names. [fn] Fisman/Luca (2018). [/fn]

In terms of gender discrimination, there is growing evidence that women in on-line platform work face several forms of discrimination. Ebay found that for similar products men were receiving more remuneration than women. [fn] Ibid. See also Gurumurthy/Chami/Alemany (2018). [/fn]

There is a need for regulatory intervention to prevent discrimination based on AI and machine learning. In France, for instance, the legal framework prohibits the use of gender, ethnic or religious individual information in data collection and application. Designers and platform companies need to acknowledge the need for algorithmic audits and corrections. AI-agility for equality and non-discrimination could become part of company labelling or certification.

---

Democracy and human rights at risk

Data, AI and ML challenges are directly related with democracy and freedom of expression for various reasons. First, participation in social networks promotes binary thinking - liking or not liking an idea, rather than nuanced interpretations. Social and political polarization is part of dominant business models of the platformized economy (e.g., Facebook, Instagram, etc.). If social networks and their algorithms succeed in this kind of polarized business model, peace and democracy can lose ground.

Second, data has increasingly become the gateway to our world, our money and our vote, [fn] Domingos (2017), p. 272. [/fn] and is today the basis of algorithmically targeted electoral marketing campaigns. This marks a new point of departure for populism and public and mass manipulation.

Third, data and AI provide a new technique for potential state interference with democracy and privacy rights, freedom of expression and social mobilization. Data generated by citizens (through a record of every item of news they read, every text they see, every posting they “like”) can be used to penalize citizens and violate their human rights. This is true across the world, from liberal to illiberal democracies and totalitarian countries. National security becomes the predictable bogeyman that is used to trample individual rights.

Fourth, platform companies’ responsibility for constitutional and human rights violations, including actions promoting violence against targeted people or groups of people, based on their use or the use of the data they gather, process and sell is not clearly understood, nor regulated. Extreme speech in the digital context is a serious concern for the future of democracies.

Last but not least, some of the algorithms and prediction models of platform companies violate national constitutional guarantees against discrimination. Legal responsibilities across the globe of the biggest monopolies of the digital economy is an emerging area for national regulation and international human rights, but there is little movement, if at all, on this vital front.

Just and equitable development in the age of AI

Data researchers in a recent journal have pointed out that “big data can make a contribution to the SDGs, but their development needs to be carefully managed to ensure they promote inclusive and participatory development”. [fn] Fukuda-Parr/McNeill (2019). [/fn] The need for action is particularly urgent in the case of decision-making systems that affect people’s well-being and freedom. [fn] Smith/Neupane (2018). [/fn] There are two primary imperatives. The regulatory wherewithal for personal data protection and the right to equality and non-discrimination, and governance frameworks for building a fair and just local to global AI-led economy. The current system for data protection is hugely under developed in much of the world, even though big data initiatives have proliferated at global, regional and national levels. UNCTAD, among others, proposes that “instead of pursuing multiple initiatives, it would be preferable for global and regional organizations to concentrate on one unifying initiative or a common smaller number of initiatives that are internationally agreed”. [fn] UNCTAD (2017), p. 94. [/fn]

UNCTAD also recommends that certain prerequisites are necessary for developing countries before any new e-commerce rules are negotiated: 1) availability of digital infrastructure, 2) affordable Internet access, 3) digital literacy, and 4) national digital policies, in particular regarding how data can enable development; sharing the revenue from monetization of data; protecting local businesses from large international players; taxation of the digital economy; income distribution and inequality; the effects of digitization on jobs, work and social security systems. Global measures are also needed to tackle inter alia, the concentration of the digital economy arising from network effects and economies of scale; abuse of dominant market power; and to check current tax optimization strategies of digital corporations.

In fact, without a serious shift in the international rules and governance arrangements of data and AI, platform companies can undermine SDGs’ advances and human rights at different levels. As UNCTAD Deputy Secretary Durand alerts:

data offers new opportunities to build knowledge and profits. However, regulators must ensure the benefits are spread evenly and that people’s privacy is protected. If not, there is a significant risk that the data-driven economy will be an increasingly unfair economy. [fn] Ibid., p. 3.    [/fn]

The key governance question in the global digital economy concerns the ownership and control over data – Who should control the intelligence of citizens in a city, in a school, or health system? Who should own and control civic intelligence? Who should ensure that rules are set for the benefit of all?

Self-regulation of internet companies will not work. [fn] Curbing Corporate Power Alliance (2019), p. 8. [/fn] To regulate AI and the new digital era first and mainly through e-commerce trade agreements, be they plurilateral, multi-country or bilateral, will not work either. The Internet Governance Forum (IGF) as a multi-stakeholder space has the potential to advance in this arena, but it is not making any rules. There is an increasing risk of a small group of countries making the rules on data from the vantage of trade deals. As trade and investment expert Jane Kelsey suggests, electronic commerce, or digital trade, is the newest and most far-reaching of the 21st century ‘new issues’ in international trade negotiations. The ‘disciplines being developed extend far beyond any legitimate notions of trade. They seek to impose global rules on governance of the digital domain – one of the most complex, multi-dimensional and hence controversial subject confronting states and societies this century. [fn] Kelsey (2019). [/fn] Moreover, the new North American Free Trade Agreement (NAFTA) II limits “governments’ ability to require disclosure of proprietary computer source code and algorithms, to better protect the competitiveness of digital suppliers” [fn] See https://ustr.gov/about-us/policy-offices/press-office/fact-sheets/2018/october/united-states%E2%80%…. [/fn] and “provides a firm foundation for the expansion of trade and investment in the innovative products and services where the United States has a competitive advantage”. [fn] Ibid. [/fn] The government of Mexico, which has some comparative advantages in relation to other developing countries in the digital economy, accepted this US condition, which limits governments’ ability to reduce code and algorithm non-transparency and discriminatory practices and to investigate anti-competitive practices, human rights violations, or fraud. In NAFTA II, coders and designers have more power than governments and their anti-discriminatory laws, as competitiveness of digital suppliers comes first. [fn] Similar clauses were incorporated in the Electronic Commerce of the Trans-Pacific Partnership Agreement (TPPA), which remained unchanged in the so-called Comprehensive and Progressive Agreement for Trans-Pacific Partnership (CPTPP) or TPPA-11. [/fn]

In a recent debate Joseph Stiglitz warned that:

we are gradually beginning to realize the wide set of problems that these digital behemoths represent for our society, in terms of privacy, market power, manipulation, fake news, a whole set of issues. And there are real efforts going on …. But, what is very clear is that none of these go far enough. And what I see is exactly what you see; that big corporations want to embed in international agreements, a framework that would stop domestic legislation. [fn] Stiglitz (2019). [/fn]

Kelsey adds that “global e-commerce rules developed by transnational corporations for their own benefit will greatly amplify threats to economic sovereignty, and disempower government to regulate digital technology to protect workers”. [fn] Kelsey (2019). [/fn]

We need what IT for Change calls “an agile legal and policy framework to curb platform excess”, to govern the new economy and its effects on the society, citizens, institutions and politics in this digital era. States should mandate ¨that private platforms in key sectors share critical data they collect with state agencies, with safeguards for protecting user and citizen privacy. To support essential public services, like city transport or health care, companies must be obligated to open up such public data for the public interest. [fn] IT for Change (2018a). [/fn]

In the 2018 Spotlight Report, Roberto Bissio stated that “recognizing knowledge and the Internet as a global public good should imply a multilateral approach, which can only be based on the primacy of human rights and the recognition of sovereignty”. [fn] Bissio (2018); for principles of ethical governance see also Winfield/Jirotka (2018). [/fn]

With the increasing additional complexity of the need to regulate AI multilaterally in its connection with very diverse fields and rights, regulation needed is inter alia on the level of the laws governing artificial intelligence uses and design, data protection, corporate liability, taxation, labour rights of the new forms of work, social security, companies’ obligation in the era of automation, competition and platform monopolies, and e-commerce, among others.

The UN recognized it “has an important role to play in supporting its Member States and other stakeholders to address new policy and normative challenges, in particular those directly affecting the central Purposes and Principles” of the UN and “for which collective global responses are necessary.” [fn] UN Secretary-General (2018), pp. 9-10. [/fn] The UN Secretary General set up the High-Level Panel on Digital Cooperation, that proposes several ideal visions. Some members of the panel propose “a human-centric world of digital technologies, with individuals retaining agency and choice among increasingly intelligent systems,” while other members “hope to see governments develop their capacity to better manage the impact of digital technologies.” [fn] See https://digitalcooperation.org/meeting-summary-consultation-insights-next-steps/. [/fn] The Panel is made up of prominent digital company representatives [fn] In addition to the co-chairs, it includes representatives from Google, Ebay, Adriel AI (partner of Google) and ABRY Partners. [/fn] that are calling for inclusive governance of digital technologies.

However, inclusive governance may mean very different things to different people. It may not amount to the accountability of actors upon which real inclusion that respects and promotes people’s rights is predicated. The High-level panel is led by two chairs (common practice), but, uncommonly, both of them come from two of the most important transnational corporations of the digital economy, Jack Ma from Alibaba and Melissa Gates, who is too close to Microsoft to be categorized in this particular panel only as a philanthropist. It is ironic to see how the concept known as ’conflict of interest’ has been forgotten in so many places, including the UN. The High-level panel may come up with interesting recommendations from their consultations, but public leadership has not been ensured, and corporate interests that prevent a truly multilateral framework for digital cooperation, are leading it.

If the international community continues to merely observe how monopolies are owning people’s data and using AI without any correction to their abusive practices and biases, existing structural asymmetries will be reproduced also in the way data and AI will be governed or ungoverned.

What is needed is an international digital development framework with policy space for developing countries to ensure that they can obtain economic value from the data that their citizens are generating. Developing countries need to be able to adopt economic and digital industrialization policies akin to those that countries of the global North successfully used in their industrialization. [fn] See James (2019). [/fn]

Evolving an effective corporate tax regime in the platform economy is challenging for two main reasons. One, the virtualisation of commercial transactions enables powerful transnational corporations to easily shift profits from higher tax jurisdictions to lower tax jurisdictions, thereby eroding the tax base of the former contexts. And two, traditional taxation regimes do not adequately account for the contribution of intangible data resources extracted from a jurisdiction for revenue generation in platform business models. This has led to calls for the basis of taxation to be shifted from “national physical presence” to a “significant economic presence” as far as the new firms of the digital economy are concerned. As the OECD (2019) has highlighted, the “significant economic presence” of digital-age business in a particular jurisdiction has to be determined through criteria such as: the existence of a user base and associated data input; the volume of digital content derived from the jurisdiction; and sustained marketing and sales promotion activities. [fn] OECD (2019), p. 16. [/fn] The government of France introduced a draft regulation for a digital services tax in March 2019, as part of shifting towards such a taxation regime grounded in the substantive economic presence logic.

There is an emerging multilateral effort towards a Legally Binding Instrument to regulate in international Human Rights law the activities of transnational corporations and other business enterprises that can give some ground to regulate data and AI. Article 4 of the current draft explicitly mentions “including activities undertaken by electronic means, that take place or involve actions, persons or impact in two or more national jurisdictions.” [fn] UN OHCHR (2018). [/fn] Thus, the Instrument would apply to platform activities, but many aspects of their operations are still not fully understood, and future drafts should ensure that platform companies and upcoming particularities of digitalization, data, AI, prediction models and remote influence are incorporated. Apparently, there is little interest to regulate transnational private companies from a human rights perspective, but their power in the real economy, reconfigured as it is through digital power, should be regulated sooner than later.

As the International Development Research Centre (IDRC) has warned, it is urgent to study the impact of AI on human rights: at a broad level, the UN recognizes that offline rights apply online, testifying to the relevance of analogue rights in digitally mediated environments. But, we need full consideration of human rights in the context of AI design and operation. The international community responded to infrastructure and investment abuses in the past through the imposition of mandatory environmental, social and increasingly gender impact assessments for certain projects. An important underlying principle is that it should always be possible to find out why an autonomous ML system made a particular decision (especially if that decision has caused or might cause harm). [fn] Winfield/Jirotka (2018), p. 8. [/fn]

Tailoring and requiring “impact assessments to the risks of AI would help encourage development programs to incorporate AI technology in ways that respect and promote human rights, including privacy, equality, and freedom of expression.” [fn] Smith/Neupane (2018). [/fn]

National imperatives

Many developing country governments (and other governments that are not among the first movers on AI) ignore the profound risks and technicalities of the expanded use of AI for almost everything. They may be inclined to focus mainly on its opportunities, beholden to the hyperbolic discourses that accompany the opportunity rhetoric.

An emerging research agenda connecting AI and human development calls for further exploration of new approaches to address liability, accountability, and redress for AI decision-making. This means, according to the IDRC, that we need to

design regulatory systems and frameworks to determine liability and accountability for AI decision-making that is erroneous, biased, or discriminatory, and establish mechanisms for redress. Measures may include policies that stipulate transparency for automated decision-making, evaluative procedures to determine the competency of AI systems, and certification of AI systems that engage in tasks requiring a degree of skill or training. [fn] Smith/Neupane (2018). [/fn]

In terms of effective regulatory models, IT for Change and others have documented and assessed existing data or AI regulatory models developed to deal with the emergence of new AI-driven risks. Public interest analysis and research is needed to contribute to an understanding of how activities such as predictive policing may be regulated, and how existing regulation needs to be adapted or new regulation developed.

In addition, all around the world, we need to update antitrust laws, to take action against platform companies’ market abuse faster and more effectively. [fn] Zimmermann (2019). [/fn]

Countries need to update national regulatory frameworks in all related areas with data, AI and ML, and need to legislate transnational companies’ rights violations and make clear links of digital rights abuses with the human rights international standards and obligations, and existing constitutional rights.

Moreover, data and AI governance implies international standards for States as duty-bearers, with the intrinsic challenge of their own use of data and AI for public policy design and vigilance. Thus, it is necessary to continue to understand how to protect citizens from rights violations in the digital era and how to avoid the erosion of civic, political, economic, social and cultural rights behind hidden algorithms and ML in hands of digital private powers and decision-makers.

The international community needs to work towards an overall paradigm shift where there is a convergence of the liberal paradigm (open AI, open internet, etc.) with a more progressive paradigm (communitization of the digital world) based on human rights and a clear norm setting on digital rights and obligations.

While values are needed, so too are norms. [fn] Ibid. [/fn] The current status of AI governance must be reshaped; if it is not, it will contribute to more being left behind. The United Nations is the forum where AI must be understood and governed as a crucial condition for human rights, democracy, peace and sustainable development. However, any process in this sense under the UN has to be led by governments with broader participation, ensuring that it is not led by platform companies’ interests, and that it is not regulated only as a matter of e-commerce or trade as currently seems to be the case.

---

Cecilia Alemany is Member of Executive Committee of Development Alternatives with Women for a New Era (DAWN). Anita Gurumurthy is a founding member and executive director of IT for Change (ITfC).