The contributions to human history made by women, children and people who speak nonstandard English will be underrepresented by chatbots like ChatGPT.
During my last appearance on the Karen Hunter Show, Hunter revealed that ChatGPT could not tell her how Blues singer Bessie Smith had influenced gospel singer Mahalia Jackson. Instead, the bot provided Wikipedia-style biographical information on each woman, but could not discuss the relationship between the two.
“There is no concrete evidence that Bessie Smith had a direct influence [on Mahalia Jackson],” ChatGPT chirped at Hunter.
“Mahalia Jackson was absolutely influenced by Bessie Smith,” Hunter corrected the bot. “She used to listen to her records as she cleaned floors! You must add that to your AI.” (Listen to Hunter discuss it here in full, from 08:16–08:55.)
If ChatGPT does not know this much about Bessie Smith—a pioneering singer who influenced generations of blues, jazz and rock musicians—what does it know?
Smith is credited with being the godmother of popular music. She stormed onto the public stage with her first single “Downhearted Blues” 100 years ago, in February 1923. Her rendition of this song in an audition with Columbia Records was so outstanding, she was signed on the spot: $1,500 in exchange for 12 records. “Downhearted Blues” reportedly sold 780,000 copies in its first year.
Smith was a pioneer in what was called “race records,” a genre of music geared towards Black American audiences. The popularity of her records made her the first pop star in American history—and made her immensely rich. Historians say at the pinnacle of her career she was earning $1,500 to $2,500 per week (approximately $26,000–$43,000 a week in today’s money).
Smith’s success resulted in her influencing hundreds of musicians, including (but not limited to) Billie Holiday, Janis Joplin and Elvis Presley. Her music continues to be so influential that three of her records were entered into the Grammy Hall of Fame: “Empty Bed Blues” in 1983, “St. Louis Blues” in 1993 and “Downhearted Blues” in 2006. Smith herself was entered into the National Women’s Hall of Fame in 1984, and in 1989 she became the fifth woman to be admitted in the Rock and Roll Hall of Fame.
Yet the ChatGPT bot was not aware of this. The question is: Why?
The answer lies in how ChatGPT acquires knowledge. ChatGPT software is taught how to interact with human beings through an engineering protocol called machine learning. During the machine learning process, ChatGPT algorithms are exposed to millions of examples of text exchanges between human beings and programmed to detect patterns in online speech and give each exchange a score. For example, the prompt “hello” followed by “how are you?” may be detected 80 percent of the time among native English speakers, but the prompt “what’s up” and the response “chillin” 69 percent of the time.
The patterns are used to build statistical models, which enable ChatGPT to predict how to respond to prompts from human users. The larger the training dataset, the better for the chatbot because it can make more accurate predictions.
This is no problem for Open AI, the company that developed ChatGPT. In 2020 they published an academic paper, asserting that their language model, which has 175 billion parameters, is the largest language model ever created. A model this size needs inputs from people across the globe, across time.
The problem is these inputs reflect the biases of their writers. This means the contributions to human history made by women, children and people who speak nonstandard English will be underrepresented. This underrepresentation was reflected in ChatGPT’s inability to discuss the enormous influence Bessie Smith has on contemporary American music.
The way we develop ‘smart’ products results in misogynoir being encoded into AI technologies—the latest being ChatGPT.
Here comes the racism and sexism.
In order for the Open AI team to train ChatGPT on Smith’s influence, large amounts about her life would have to be available to engineers to use in the training data. Their goal is to create a chatbot that can be used across the globe so there is no incentive to leave out data. However they rely on musicologists whose job it is to research and produce new knowledge about musical production across the globe, to write books and articles in credible publications about Smith’s work. The problem is that in order for musicologists to do this, scholars within this field would have to acknowledge the power Smith yielded in shaping the American cultural landscape. However making such an admission would require them to admit Black women were challenging what bell hooks calls the white supremacist, patriarchal capitalistic values which place middle-class, cisgendered, heterosexual, able-bodied white men at the center of American culture. Therefore Smith’s contributions have been minimized.
This erasure of Black women from American history is driven by misogynoir—a term coined by media scholar Moya Bailey and used to describe how Black women become stigmatized and pathologized when they become “too successful” by the standards of white society. Policies and practices are then developed to “keep them in their place,” whether it is paying them less than white women, providing inadequate maternal healthcare, or (in Smith’s case) ignoring their contributions to American society.
The way we develop “smart” products results in misogynoir being encoded into AI technologies—the latest being ChatGPT.
ChatGPT’s ability to engage in the expression of misogynoir may not be a concern for some. But Microsoft plans to use ChatGPT to disrupt the online search experience by integrating it into its search engine, Bing. We rely on search engines to deliver accurate, objective, unbiased information—but this is impossible, due to biased training data and the algorithms that drive ChatGPT, which are designed to predict rather than fact-check information.
ChatGPT could soon become a tool used to weaponize mis- and disinformation against women. We all remember how lies about game developer Zoe Quinn trading sex for video game reviews resulted in her becoming the center of Gamergate—a years-long online harassment campaign levied against women in the gaming industry. There is nothing to stop ChatGPT bots being used to do this—not only do they not check facts, but also they’re not regulated by the federal government.
I love technology and love it when new applications hit the market. My only wish is that AI design protocols are shaped by the Black feminist principles outlined in the Combahee River Collective Statement—which states, “Black women are valuable.” Engineers must ensure Black women, and all marginalized people, are fairly represented in their datasets.
Imagine a chatbot designed to write women, Black people, people with disabilities and non-binary femmes into global history. Imagine how including them would improve the functionality of AI technologies, and imagine how this would change public perceptions of those who are presented as problems rather than the solutions for the problems we face. This is a tech sector that would serve all people including women and non-binary femmes. This is the tech sector we all deserve.
Up next:
U.S. democracy is at a dangerous inflection point—from the demise of abortion rights, to a lack of pay equity and parental leave, to skyrocketing maternal mortality, and attacks on trans health. Left unchecked, these crises will lead to wider gaps in political participation and representation. For 50 years, Ms. has been forging feminist journalism—reporting, rebelling and truth-telling from the front-lines, championing the Equal Rights Amendment, and centering the stories of those most impacted. With all that’s at stake for equality, we are redoubling our commitment for the next 50 years. In turn, we need your help, Support Ms. today with a donation—any amount that is meaningful to you. For as little as $5 each month, you’ll receive the print magazine along with our e-newsletters, action alerts, and invitations to Ms. Studios events and podcasts. We are grateful for your loyalty and ferocity.
