The social and economic impact of the COVID-19 pandemic has been wide and far-reaching. In particular, it has had a disproportionate effect on poorer households. Some of the hardest-hit industries, such as retail and food service, historically employ lower-wage workers that now find themselves without jobs or emergency funds to weather the storm. There’s even research data that suggests inequality contributes to higher death rates among COVID-19 patients with lower economic status.

Economic inequality is a complex and global issue, and it is growing — even in the developed world — as a report released by the United Nations in January found. Furthermore, the results of a recent survey by the University of Chicago suggest we’re likely to see the gap between poor and affluent grow even larger as a result of COVID-19.

The global economy is evolving rapidly and we need better methods to design and evaluate economic policy. Economists need to be able to identify faster and more comprehensive alternatives to find tomorrow’s solutions. This is where we believe that AI technology can be used for good.

Introducing the AI Economist

The Salesforce Research team recently released a tool called the AI Economist as a first-of-its-kind machine learning model that aims to improve equality and productivity through the creation of AI-driven tax policy recommendations. The AI Economist uses a two-level reinforcement learning (RL) framework and is highly flexible, designed to optimize for any social objective (e.g., equality, productivity or sustainability), that can be set by the user.

We compared the AI Economist with three other baseline tax methods:

• The free market with no taxation or redistribution
• A progressive tax mirroring the 2018 United States federal tax schedule (i.e., marginal tax rates increase with income)
• An analytical tax model proposed by economist Emmanuel Saez, which in our setting leads to a regressive tax schedule

In our simulations, the AI Economist achieved a 16% gain in the trade-off between equality and productivity compared to the next best framework, the Saez model. Compared to the free market, the AI Economist also improves equality by 47 percent, with only an 11% decrease in productivity.

We believe these initial results demonstrate the potential of applying a data and simulation-driven approach to quickly create equitable and effective economic policies.

Applying Reinforcement Learning to Optimize Economic Policy

Reinforcement learning algorithms use smart trial-and-error strategies in an environment to optimize policy models for a specified goal. During this process, the learning algorithm continuously uses feedback it receives to improve the policy models.

You have likely read about some of the higher-profile applications of RL. It’s been the go-to method that enabled AI to compete and even win against humanity’s best in popular games such as Go, Dota 2 and Starcraft.

What is different about the AI Economist is that the goal is not adversarial. The objective is not to beat the other team. In this case, “victory” is achieved when the AI creates a socially optimal balance between equality and productivity.

The AI Economist uses RL at two levels. At the first level, it uses a collection of AI agents to simulate how people might react to different taxes. The AI agents earn money by collecting and trading resources in order to build houses. These agents maximize their utility and happiness by adjusting their behavior using RL algorithms. At the second level, a social planner uses RL to adjust and evaluate the impact of tax policies on the world.

Why Taxes?

Taxes are a common tool for national and local governments to redistribute wealth, and it is a near-universal part of society. We have seen it in action recently as subsidies funded by taxes help those impacted by the pandemic, particularly in the U.S.

But no one has truly determined how tax policy can be feasibly optimized in complex dynamic economies. Creating effective economic policy is complicated due to the near-infinite number of possible contingencies to consider. This is why our team of AI engineers and scientists attempted to crack the code, and why we enlisted the help of experts such as Professor David Parkes, who heads the Economics and Computer Science group at Harvard University.

Together, we built a framework that can simulate millions of years of economies and evaluate tax policies in a fraction of the time it would take to run a real-world economy. Careful real-world experimentation with taxes is almost impossible, so viable economic simulation environments can help find effective solutions faster.

Our simulations saw AI agents react strategically to tax policy changes. In some cases, agents changed their behavior to game the system and lower their effective taxes. The AI tax model adapted well in response to those behavioral changes and was still able to improve equality among agents.

To help us ensure we’re on the right path, we even took the first steps to evaluate our economic policies with real people. When we tested a version of the AI tax model with actual people earning money in the economic simulation, it achieved similar or better social welfare improvements compared to baselines.

Going Forward

Ultimately, our vision for the AI Economist is for it to be a step forward for AI-powered economic models that support real-world policymaking to improve social welfare. Our results indicate that there is real potential for this to be a valuable tool for policymakers.

So as a next step, the economic simulation is now available via – to anyone interested in helping the framework expand. With the help of a broader community with more academics, economists, policymakers and AI engineers, we look forward to seeing the AI Economist play a real role in creating more impactful economic policies that benefit everyone.