I'm going to move on to the
to introducing Shari Trewin
and she is an engineering manager at Google
and leading a team that develops assistive technologies.
So I'm really looking forward
to your vision of and
how what's next?
What's the future holding for us in assistive AI.
So as we had yesterday
at the end of the keynote Jutta
will join us and we'll have this
even more interesting conversation between Shari Trewin
making it really appetizing for the keynote.
So Shari the floor is yours.
All right. Thank you very much, Carlos.
I just.
Okay. Can you hear me okay?
Yes. Good.
All right.
What a pleasure it is to participate in this symposium and
hear from our opening keynote, Jutta,
and all our panelists over the last two days.
Thank you so much for inviting me.
It's my privilege to finish us up now.
So yesterday Jutta grounded us
all in the need to do no harm and talked about some of the ways
we can think about detecting and avoiding harm.
Today, I'm going to focus on digital accessibility applications
of AI in general and ask where next
for assistive AI?
I You see my screen?
Yes, we are. You are. Okay.
I just didn't show mine all good.
So my name is Shari Trewin.
I'm an engineering manager in Google's accessibility team.
I'm also the past chair of the ACM's SIGAccess
Special Interest Group on accessible computing.
My background is computer science and AI, and I've been thinking about the ways
that AI plays into accessibility for many years.
Much of my work in thinking on AI and the AI fairness was done
when I worked at IBM, where I was program director for IBM Accessibility.
So shout out to any IBM friends in the audience.
At Google,
my team has a focus on developing new assistive capabilities
and as we've been discussing for the last few days,
AI has an important role to play.
So there's been a lot of buzz in the news lately.
Both excitement and alarm about generative AI,
especially these large language models.
So for example, the ChatGPT model from OpenAI
has been in the news quite a bit,
in case you haven't played with it yet, here's an example.
So I asked ChatGPT how will AI change digital accessibility?
Let's try to get it to write my talk for me.
And it responded with a pretty positive viewpoint.
It said AI has the potential to significantly
improve digital accessibility for people with disabilities.
Here's a few ways that AI could contribute to this goal.
It went on to
list four examples of transformative AI.
All of these have been major topics at this symposium, and for each one
it gave a one or two sentence explanation of what it was
and who it's helpful for.
Finally, it concluded that AI has the potential
to make digital content and devices more accessible to people
with disabilities, allowing them to fully participate in the digital world.
It seems pretty convincing and well written.
Perhaps I should just end here and let AI have the last word. But,
you know, it's kind of it's kind of mind blowing,
although it was pretty terrible jokes and this is what it can do
without explicitly being connected to any source of truth.
But it does get things sometimes flat out wrong.
And there's a risk of bias in the training data being reflected in the predictions.
And this limits the
ways that we can apply this technology today.
But it also gives us a glimpse into the future.
I'm not going to take medical advice
from a generative AI model yet, but as we get better
at connecting this level of language fluency with knowledge,
improving accuracy, detecting and removing bias,
this opens up so many new possibilities for interaction models
and ways to find to consume information in the future.
So I'll come back to that later.
For today's talk, I'm going to
slice the topic a little bit differently.
I want to focus on some of the general research directions
that I see as being important for moving digital
accessibility forward with AI.
So in our opening keynote, Jutta laid out
some of the risks that can be associated with AI.
It's not created and applied with equity
and safety in mind, and it's important
to keep these considerations in mind as we move forward with A.I.
When the benefits of
AI do outweigh the risks in enabling digital access,
we still have a way to go in making these benefits
available to everyone, in fact, to make them accessible
so start by talking about some current efforts
in the direction making assistive
AI itself more inclusive.
The second topic I want to cover is where
we choose to apply AI.
Focusing on why I called it
AI at source and find web
accessibility work in role emphasizes
the need to shift left that is, to bake accessibility
in as early as possible in the development of a digital experience.
So I'll discuss some of the places where AI can help with that shift
left and highlight both opportunities and important emerging challenges
that we have for web accessibility.
So we know the AI has already changed
the landscape of assistive technology.
So one research direction is how do we make these AI models more inclusive?
And I want to
start with a little story about captions.
In 2020, I was accessibility chair
for a very large virtual conference.
We provided a human captioner
who was live transcribing the sessions in a in a separate live feed.
So I'm showing an image of a slide from a presentation
here with a transcription window to the right.
I spoke with a hard of hearing attendee during the conference
who used captions to supplement what he could hear,
and he told me while the live feed had quite a delay.
So he was also using automated captions
that were being streamed through the conference provider.
Let's add them to this view highlighted in Green.
So these had a little less delay, but they had accuracy problems,
especially for foreign speakers or people with atypical speech,
and especially for people's names or technical terms.
The important parts.
So he also turned on the automated captions in his browser,
which used a different speech to text engine.
I've added those on the screen to
and supplemented that with an app on his phone
using a third different speech recognition engine, capturing
the audio as it was played from his computer and transcribing it.
So that's four sources of captions to read.
None of them was perfect, but he combined them to triangulate
interpretations where the transcripts seem to be wrong.
So we could say AI
powered captions were helping him to access the conference, no doubt about it.
But it wasn't a very usable experience.
He was empowered, but he also had a huge burden
in managing his own accessibility.
And there were still gaps, though,
as Michael Cooper pointed out yesterday, imperfect captions and descriptions
can provide agency but can also mislead users
and and waste their time.
I also want to point out that this particular user
was in a really privileged position because he knows about all these services.
He has devices powerful enough to stream all these channels,
has good Internet access.
He has a smartphone, he has the cognitive ability
to make sense of this incredible information overload.
This really isn't equitable access.
Right.
And and the captions themselves weren't providing equitable
representation of the conference speakers
to those with atypical speech were at a disadvantage
in having their message communicated clearly.
So there's an important gap to be filled.
One of the current limitations of automated captions is poor
transcription of people with atypical speech,
and especially when they're using technical or specialized language.
So for example, Dimitri Kanevsky
is a Google researcher and inventor.
He's an expert in optimization and algebraic geometry, among many other
topics.
He's Russian and deaf, both of which affect his English speech.
I'm going to play a short video clip of Dimitri.
So, Dimitri said
Google has very good general speech recognition,
but if you do not sound like most people, it will not understand you.
And on the screen a speech engine translated that last part of the sentence
as. But if you look at most of people it will look
and defend you.
So, people
with disabilities that impact speech such as verbal palsy,
stroke, Down's Syndrome, Parkinson's, ALS are also impacted
by lack of access to speech recognition, whether it's for controlling a digital
assistant, communicating with others, or creating accessible
digital content. Oh,
I want to go to the next slide.
So Google's Project Euphonia set out to explore
whether personalized speech recognition models can provide accurate speech
recognition for people like Dimitri with atypical speech.
And this is a great example of the way research
can help to move the state of the art forward.
So the first challenge, as many people
have mentioned already today, was a lack of suitable speech data.
Project Euphonia
collected over a million utterances
from individuals with speech impairments.
And then the researchers built individual
models for 432 people and compared them
to state of the art general models.
They found that the personalized models could significantly reduce
word error rates, and so the error rates had gone from something like 31%
with the general models down to 4.6%.
So it's not just a significant improvement,
but it's enough of improvement.
It gets to a high enough point to make the technology practically useful.
And in fact, they've even found that these personalized models
could sometimes perform better than human transcribers for people
with more severely disordered speech.
So here's an example of Dimitri
using his personal speech recognition model.
You can see the.
So the transcription this time is make all voice interactive
devices be able to understand any person speak to them.
It's not perfect, but it's much, much
more useful.
Project Euphonia has started in English, but it's now expanding to include
Hindi, French, Spanish and Japanese.
So that project demonstrated how much better
speech recognition technology could be.
But the original data wasn't shareable outside Google
and that limited the benefits of all that data gathering effort.
So the Speech Accessibility Project at the University of Illinois
is an example of of what we might do about that problem.
It's an initiative to make a dataset for broader research purposes.
It was launched in 2022, and it's a coalition
of technologists, academic researchers and community organizations.
The goal is to collect a diverse speech dataset for training
speech recognition models to do better at recognizing atypical speech.
It's building on some of the lessons learned in Project
Euphonia, paying attention to Ethical data collection.
So individuals are paid for participating.
Their samples are de-identified to protect privacy.
The dataset is private.
It is managed by UIUC
and will be made available for research purposes,
and this effort is backed by cross-industry very broad support
from Amazon, Apple, Google, Meta and Microsoft.
It's going to enable both academic researchers
and industry partners to make progress.
Although the current work is focused on
speech data, this is in general a model that could be used for other data
that's needed to make models more inclusive
so we could think of touch data.
And there are already significant efforts going on together.
Sign language video data for for sign language translation.
Then Project Relate is an example of the kind of app
that can be developed with this kind of data.
It's an Android app that provides individuals
with the ability to build their own personalized speech models
and then use them for text to speech, for communication,
for communicating with home assistants.
Personalized speech models look really promising,
and potentially a similar approach could be taken into build
personalized models for other things like gesture recognition,
touch screen interactions, interpreting inaccurate typing.
I think there's
there's a world of opportunity there that we haven't really begun to explore.
So now that we know we can build effective personal models
from just a few hundred utterances, can we learn from this?
How to build more inclusive general models would be a really important goal.
Can we improve the performance even further
by drawing on a person's frequently used vocabulary?
Can we prime models with vocabulary
from the current context?
And as Shivan Sing mentioned yesterday, we're beginning to be able to combine
text, image and audio sources to provide
a richer context for AI to use.
So there's very fast progress happening
in all of these areas.
Just as another example, the best student paper at the Assets 2022 conference
was using vocabularies that were generated automatically from from photographs
to prime the word prediction component of a communication system,
for a more efficient conversation around those
photographs.
Finally,
bring your own model.
I really agree with Shaomei Wu when she said yesterday
that use cases of media creation are under investigated,
we can apply personalized models in content creation.
Think about plugging in your personal speech model
to contribute captions for your livestreamed audio for this meeting.
The potential is huge and web standards might need to evolve
to support some of these kinds of use cases.
Okay.
Next part. When we when we talk about assistive AI,
we're often talking about technologies that are
that are being applied at the point of consumption, helping an individual
to overcome accessibility barriers in digital content or in the world.
And I want to focus this section on AI at source and why that is so important.
Powerful AI tools in the hands of users
don't mean that authors can forget about accessibility,
and we've been talking about many examples of this through this symposium.
But here are a few that appealed to me.
So I'm showing a figure from a paper.
The figure is captioned user response time
by authentication condition, and the figure itself is a box plot.
It shows response times from an experiment for six different
experimental conditions.
So it's a pretty complex figure.
And if I'm going to publish this in my paper, my paper is available
and I need to provide a description of this image.
There's so much information here.
When faced with this task,
about 50% of of academic authors
resort to simply repeating the caption of the figure.
And this is really no help at all to a blind scholar.
They can already read the caption that's in text.
So usually the caption is saying
what information you'll find in the figure, but it's not giving you
the actual information that's in the figure.
Now, as we discussed in yesterday's panel,
that blind scholar reading my paper could use
AI to get a description of the figure, but AI doesn't
really have the context to generate a good description.
Only the author knows what's important to convey.
At the same time, most authors aren't familiar
with the guidelines for describing images like this,
and writing a description can seem like a chore.
That's why I really love the ideas that Amy Powell shared yesterday
for ways that a tool could help content creators with their own description tasks,
perhaps by generating an overall structure
or an initial attempt that a person can edit.
I mean, there are
there are existing guidelines for describing different kinds of chart.
Why not teach AI how to identify different kinds of chart
and sort of generate a beginning description?
And Shivam Singh was talking yesterday as well about recent progress in this area.
So ideally AI could refine
its text in an interactive dialog with the author
and then the resulting description would be provided in the paper
and anyone could access it whether or not they had their own AI.
So that's what I mean by applying AI at source where there's a person
with the context to make sure the description is appropriate
and that can provide a better description.
Of course, it can only provide one description.
There is also an important role for image understanding that can support
personalized exploration of images
so that a reader could query information
that wasn't available in a short description.
Like what were the maximum and minimum
response times for the gesture condition in this experiment?
I'm not saying that AI at source is the only solution, but it's important
and perhaps it's an undeveloped piece.
Here's the second example.
I love examples
as we were just talking about in the earlier panel.
Text transformations can make written content more accessible.
So, for example, using literal language
is preferable for cognitive accessibility.
So an idiom like she was in for a penny in for a pound
can be hard to spot if you're not familiar with that particular idiom.
It can be very confusing if you try to interpret it literally.
Content authors might
use this kind of language without realizing
language models could transform text to improve accessibility in many ways.
And one is by replacing idioms with more literal phrasing.
So here I asked a language model to rephrase the sentence
without the idiom, and it came up with a very sensible,
although a little complex little literal replacement.
She decided to fully commit to the situation no matter the cost.
Again, this could be applied
as a user tool and as a tool for authors
to help them identify where their writing could be misinterpreted.
The one puts the onus on the consumer
to bring their own solution, apply it and be alert for potential mistakes.
The other fixes the potential access problems
at source, where the author can verify accuracy
and as I mentioned earlier, because today's large language
models are not connected to a ground truth and they do have a tendency
to hallucinate.
Applying them at source is one way to reap the benefits
much more quickly without risking harm to vulnerable users.
Once we collect language models, connect them to facts
or connect speech to the domain of discourse,
we will really see a huge leap in performance,
reliability and trustworthiness.
So in the previous two examples, AI could be applied at source.
What about when the AI has to be on the consumer side,
like when using text to speech, to read out text on the web.
On the screen
here is the start of the Google information sidebar about Edinburgh.
The capital city of Scotland. Is a heading, subheading in the main text paragraph.
Text to speech is making
huge advances with more and more natural sounding voices becoming available
and the capability of more expressive speech,
which itself makes comprehension more easy.
And expressiveness can include things
like adjusting the volume, the prosody.
When reading a heading, maybe I would naturally read it a little louder.
Pause afterwards.
For a TTS service to do the best job reading out text on the web.
It helps to have the semantics explicitly expressed.
So for example, the use of heading mark up on Edinburgh on this passage.
It's also important that domain specific terms
and people's names or place names are pronounced correctly.
So many people not from the UK on first sight,
they pronounce Edinburgh.
Web standards,
if they're applied properly, can mark up the semantics like headings
and pronunciation of specialized or unusual words, helping the downstream AI
to perform better. In fact,
AI could
also be used to identify the intended structure and compare against the markup,
or identify unusual words or acronyms where
pronunciation information could be helpful,
and then the passage can be read appropriately by your preferred text
to speech Voice at your preferred speed and pitch.
Can also be used by a speech
to text model to marry the vocabulary on the page
with what you're saying as you're interacting with the page
to use voice controls.
So I'm showing this example to illustrate the web accessibility standards
work together with assistive AI techniques to enable the best outcome. And
many uses of assistive technology can benefit from this information.
So thinking about applying AI at source,
there's an important role here for A.I.
that makes sure that the visual and structural DOM
representations are aligned.
So I just want to reiterate the
the powerful benefits of applying AI at authoring time
that these examples illustrate.
So for software, removing the burden from people
with disabilities to supply their own tools to bridge gaps.
Secondly, it benefits more people,
including those people who don't have access to the AI tools,
people with low end devices, poor internet connectivity,
less technology literacy.
Thirdly, content creator can verify the accuracy and safety of suggestions,
mitigating harms from bias or errors because they have the context
and AI can also potentially mitigate harms in other ways.
For example, flagging videos, images or animations that might trigger adverse
health consequences for some people like flashing lights.
So AI inside is likely to reach more people
than AI provided by end users.
I think this is how we'll get the most benefit for the least
harm.
It's also a huge opportunity to make accessibility easier to achieve
AI can make it much quicker and easier to generate the accessibility information
like captions or image descriptions as we've discussed,
and lowering the barrier to entry with assistive
tools is one way to encourage good accessibility practice.
AI can proactively identify where accessibility work is needed
and evaluate designs
before even a line of code has been written.
But perhaps
the biggest opportunity and the greatest need for our attention
is the use of AI to generate code.
Which brings us to the final section
of this talk.
So in previous section
we talked about ways that I can be applied
in content creation to help build accessibility in,
but AI itself is also impacting the way websites are designed
and developed independent of accessibility.
So in this section, let's think about how this change will impact
our ability to bake accessibility in and can we use
AI to help us.
As accessibility advocates
we have long been pushing the need to shift left, and by that
we mean paying attention to accessibility right from the start of a project.
When you're understanding the market potential,
when you're gathering requirement, when you're understanding and evaluating
risks, developing designs and developing the code that implements those designs.
In a reactive
approach to accessibility, which is too often what happens,
the first attention to accessibility comes when automated tools
are run on an already implemented system.
Even then, such tools
don't find all issues and
may not even find the most significant ones
which can lead teams to prioritize poorly.
So with that, our reactive approach
teams, can be kind of overwhelmed with hundreds
or even thousands of issues kind of late in their process and
have difficulty tackling it.
It makes accessibility seem much harder than than it could be.
So this morning's panel, we discussed ways that AI can be used
in testing to help find accessibility problems.
AI is also already being used earlier
in the process by designers and developers.
In development, for example,
GitHub Copilot is
AI model that makes code completion predictions
and GitHub claims that in files where it's turned on nearly
40% of code is being written by GitHub copilot in popular
coding languages.
There's also systems that generate code from design wireframes
or from high resolution mockups, or even from text prompts.
So it's incumbent on us to ask
what data are those systems trained on
in the case of copilot is trained on GitHub
open source project code.
So what's the probability that this existing code is accessible?
We know that we still have a lot of work to do to make digital accessibility
the norm on the web. Today is the exception,
and many of you probably know WebAIM does
an annual survey of the top million website home pages.
It runs an automated tool and puts the issues that it found
almost 97% of
the million pages had accessibility issues,
and that's only the automatically detectable ones.
They found an average of 50 issues per page.
And they also found the page complexity is growing
significantly.
Over 80% of the pages they looked at had low contrast
text issues.
More than half had alternative text missing for images,
almost half had missing form labels.
So even though these are issues, they're easy to find with
the automated tools that we have today, they're still not being addressed.
These are very basic accessibility issues and they're everywhere.
So we know what this will surely mean for AI models
learning from today's web.
And here's an example of how this might be playing out already.
So code
snippets are one of the most common things that developers search for,
and a large language model can come up with pretty decent code snippets.
And this is this is a game changer for developers and it's already happening.
So let's say a developer is new to Flutter.
Flutter is Google's open source mobile app development platform.
They want to create a button labeled with an icon known as an icon button.
So on this slide
is the code that ChatGPT produces when it's asked for a Flutter code
for an icon button
along with the code snippet, it also provided some explanation
and it even links to the documentation page.
So it's pretty useful.
And the code it gave for an icon
button includes a reference to what icons to use and a function to execute
when the button is pressed.
There's really just one important difference between this example
generated by ChatGPT and the example given
in the Flutter documentation.
ChatGPT didn't include a tooltip,
which means there's no text label associated with this button.
That's an accessibility problem.
Let's give it credit.
ChatGPT did mention that it's possible to add a tool,
but developers look first at the code example.
If it's not in the example, it's easily missed.
But in the training data here, it seems the tooltip was not present
enough of the time for it to surface as an essential component of an icon button.
So, you
know, there's lots of example code available online,
but how much of that code demonstrates accessible coding practices?
Given the state of web accessibility, it's likely
the answer is not much.
So our AI models are not going to learn to generate accessible code.
It's really it's just like the societal bias
of the past being entrenched in training sets of today.
The past lack of accessibility could be propagated into the future.
So here we have an opportunity and a potential risk.
AI can help to write accessible code,
but it needs to be trained on accessible code
or augmented with the tools that can
correct accessibility issues.
And I think it's important to point out as well
that I deliberately use an example in a framework
rather than HTML example, because that's what developers are writing in these days.
They're not writing raw HTML,
their writing of frameworks, and there are many, many different
frameworks, each with their own levels of accessibility and
ways to
incorporate accessibility.
So one thing
is that the theme of this morning about data being really essential
comes up here again.
Do we have training data
to train a code prediction model?
Perhaps with transfer learning to generate more accessible code?
Do we have test sets even that we can test
code generation
for its ability to produce accessible code?
So when we're developing datasets
for either training or testing,
we have to think in terms of the diversity
of of frameworks and methods that developers are actually working with,
if we want to catch those issues at the point
of creation.
Again, where, where,
AI is generating code for a whole user interface
based on a visual design, we need to be thinking about what
semantics should that design tool capture to support the generation of code
with the right structure, the right roles for each area,
the basic fundamentals of accessibility. So
a final call to action for
the community here is to think about what what do we need to do here, whether it is
advocacy, awareness raising, research, data gathering, standards
or refining models to write accessible code.
This technology is so really young.
It has a lot of room for improvement.
This is a perfect time for us to define how accessibility
should be built in and to experiment with different ways.
And, you know, in my opinion, this perhaps more than anything, is
the trend that we need to get in front of as an accessibility
community, before the poor
practices of the past are entrenched in the automated code
generators of the future.
AI is already shifting left, so we must make sure
accessibility goes with it.
So to summarize, we can broaden access to assistive AI through personalization.
To get the
benefits of AI based empowerment to all users,
we should make sure that AI integration with authoring tools
and processes is applied where it can to make it easier to meet accessibility
standards and improve the overall standard.
Born accessible is still our goal
and AI can help us get there if we steer it right. As a community
we have, we have a lot of work to do, but I'm really excited
about the potential here.
So thank you all for listening.
Thanks to my Google colleagues and the IBM
accessibility team for
feedback and ideas and great conversations.
And now I want to invite Jutta to to join
and let's let's have the conversation.
Thank you, Shari.
And I really, really appreciate your coverage of authoring
and the prevention of barriers and the emphasis on timely,
proactive measures.
There may be an opportunity actually to relook at authoring
environments, etc., within W3C.
Yeah, actually, just just to respond to that really quickly, I do wonder, like,
should we be focusing on evaluating frameworks
more than evaluating individual pages?
You know, I would we get more bang for our buck
if that was where we paid attention?
Yes, exactly.
The opportunity to add, and especially as
these tools are
now also assisting authors, which was part of what
the authoring of the authoring
standards were looking at, prompting
providing the necessary supports and making it possible for individuals
with disabilities to also become authors of code
and to produce code so the greater participation of the community,
I think, will create that some of that culture shift.
So thank you very much for covering this.
So in terms of the questions that we were going to talk about,
you had suggested that we might start with one of the thorny questions
that was asked yesterday that we didn't get time to respond to.
So the the question was, do you think that AI
and big companies such as Google and Meta driving research in AI
can be problematic with respect to social societal issues
which don't necessarily garner the highest revenue?
And if so, how do you think we can approach this?
Yeah, thank you Jutta
and thank you to the person who asked that question too.
It's true that company goals and society
can pull in different directions.
I do think there are benefits to having
big companies working on these core models because they often
have better access to very large datasets that can
bring breakthroughs that then
others can share, then that can help
raise the tide to raise all boats in a way.
But advocacy and policy definitely have
an important role to play in guiding the application of AI,
in the direction of AI research, the ways that it's applied.
Also, I wanted to say one approach here could be through initiatives
like the Speech Accessibility Project that I talked about.
So that's an example of of big tech working
together with advocacy groups and academia
to create data that can be applied to many different research projects.
And that's a model that we could try to replicate.
Do you think that that I mean, you've talked quite a bit about the opportunity
for personalization and of course, one of the biggest issues here is that
large companies are looking for the largest population,
the largest profit, which means the largest customer base,
which tends to push them towards thinking about
and not thinking about minorities, diversity, etc..
But the training models and the personalization
strategies that you've talked about
are things that are emerging possibilities within large learning models.
We have the opportunity to take what has already been done
generally and apply
more personalized, smaller data sets, etc..
Do you think there's a there's a role
for the large companies to prepare the the
the ground and then for
the remaining issues to
piggyback on that with with the new training sets?
Or do you think even there we're going to have
both
cost and availability issues?
Well, you know, I mean, I think that the model
that you described is is is already happening in places
like with the the speech accessibility project.
The ultimate goal would be
to have one model that can handle
more diverse datasets
and it takes
a concerted effort to gather that data.
But if a community gathered the data
and it was possible to contribute that data, then,
you know, that's that's another direction that we can
influence the the larger models that are trained on large data.
But personalization is
I think it's going to be very important
for for tackling some of that tail end. So
personalization is not just an accessibility benefit.
There's there's lots of
there's lots of tail populations, small
end populations that add up to a large end and a lot of people.
So the more the I think that the big companies benefit
greatly by exploring these smaller populations
and learning how to adapt models to different populations.
And then
as I mentioned, the ultimate goal would be to learn how to fold that back
in to a larger model without it being lost in the process.
Yeah, we have the dilemma that the further you are from the larger model,
the more you need to work to shift it in your direction.
So the
that is
something I think that will need to be addressed.
Whatever personalization happens, the people that need the personalization
the most will have the greatest difficulty with the personalization.
Do you think there's any strategies that that might be available
for us to use to address that particular dilemma?
Yeah You're you're touching my heart with that question
because I, I really
that's been an ongoing problem in accessibility for forever
the the and not just in the context of AI,
but the people who would benefit the most
from personalization may be in a position that makes it hard to discover
and activate even personalization that's already available.
So one approach that works, I think in
some context is dynamic adaptation,
where instead of a person needing to adapt to a system,
the system can kind of flexibly adapt to the person that's using it.
And I think that works in situations where
the person doesn't need to behave any different
to take advantage of that adaptation.
It doesn't work so well where
there's maybe a specific
input methods that you might want to use
that would be beneficial where you need to do something different.
So for, you know, for language models, maybe we can imagine
an Uber language model that
first recognizes, Oh, this person's speech is closest to this sub model
that I have learned, and I'm going to use that model for this person.
And you could think of that in terms of decreasing the
distance, yeah.
Yeah, yeah.
So that's one that's one idea.
What what do you think.
Yeah.
I'm wondering whether there is an opportunity
or if there ever will be taken an opportunity
to rethink just how we, we design
what design decision we make and how we develop
and brings the systems to market
such that there is the opportunity for greater
democratization or
access to the tools and that
don't begin with the the notion of let's
design first for the majority and then think about
I mean, this is is an inflection point.
There is an opportunity for small data sets, zero shot training,
etc., transfer, transformation transfer.
Is this a time when we can have a strategic push to say,
let's think about other ways of
of actually developing these tools and releasing these tools?
Maybe that's a little too idealistic that
I don't know what you're thinking, is there?
Yeah, I, I think especially
if you're in a domain where you've identified that there's,
you real risk and strong risk of bias.
It's a it should be a part of the design process
to include people who would be outliers, people
who are going to test the boundaries of what your solution can do.
People that are going to help you
understand the problems that it might introduce.
So it it's
it's what should happen
I think in design in in any system
that especially if you're thinking in AI, you need to think about
the risks that you might be introducing and you can’t really think about that
without having the right people involved.
Right...
Whether that's by... somebody yesterday I think mentioned
something about
teaching designers
and developers more about accessibility.
And and I think that's a really important point too
that building diverse teams is really important.
Getting more diversity into computer science is really important.
But teaching the people who are already there building things is also important.
And I don't you know, I don't meet very many
people who say, Oh, I don't care about accessibility.
It's not important.
It's it's more that it's it's still too difficult to do.
And that's one place where I think AI can really, really help.
And some of the the tools that people have talked about today are examples of that
where if we can make it easy enough and lower that barrier
and take opportunity of these creation points
to teach people as well about accessibility.
So not
not always
to fix everything for them, but to fix things with them
so that they can learn going forwards and grow.
I think that's a really exciting area.
Yes, and a great way to to support
born accessible, so accessible by default
with respect to what is the tools that are used to create it.
I'm you you contributed some questions that you would love to discuss
and one of the first ones is is AI’s role mostly considered
as improving assistive technology or digital accessibility in general.
And of course, this gets to this idea of not creating
a segregated set of innovations that specifically address
people with disabilities, but also making sure that
the innovations that are brought
about by addressing the needs of people who whose needs of,
well, who face barriers can benefit the population at large.
So do what what do you think?
What is the the future direction?
Yeah, this was a question that came from
an attendee, I think that was put into in the registration process.
And I,
I do think it's really important to view AI
as a tool for digital accessibility in general and
and not to just think about the end
user applications although those personal AI
technologies are really important and they're life changing and they can do things
that aren't achievable in any other way, but AI
is already a part of the development process
and accessibility needs to be a part of that.
And we have so many challenges to solve there.
I think it's
an area that we need to pay more attention to.
So not just applying AI to detect accessibility problems,
but engaging with those mainstream development tools
to make sure the accessibility is considered.
One sort of associated
piece to this that came to mind.
And I'm going to take the privilege of being the person asking the questions.
I mean, the focus of most of AI innovation
has been on replicating and potentially replacing human intelligence
as opposed to augmenting or thinking about other forms of intelligence.
And I wonder whether the I mean, our experiences in assistive technology
and how technology can become an accompaniment
or an augmentation rather than a replacement
might have some insights to give in this
improvement of digital inclusion.
Yeah, I think you're you're absolutely right.
It's it's human AI cooperation and collaboration
that's going to get us the best results. And
the the
language the language models
that we have, the promise that they have of more interactive
dialog like interactions
are, you know, heading
in a direction they're going to support
much more natural human AI dialog
and accessibility is such a complex
topic where
it's not always obvious
what I'm trying to convey with this image.
How important is is the thing, you know, it's not it's
not necessarily easy to
to decide what exactly is the
correct alternatives for for something
or there's plenty of other examples
where a
the combination of an AI that has been trained on
some of the general principles of of good accessibility practice
and a person who may not be as familiar but really understands the domain
and the context of this particular application.
It's when you put those two things together
that, things are going to start to work
so AI can support the person, not replace the person.
And of course,
the the
one issue that we need to thorny issue that we need to overcome
with respect to AI, is that
the challenge of addressing more qualitative
non quantitative values and ideas etc.,
so that it'll be interesting to see what happens there.
Yeah. Yeah.
I thought Yeliz had a very good suggestion this morning of
perhaps we should pay attention to
how people are making these judgments, how the AI,
how the accessibility experts
make these judgments, what are the principles and,
and can we articulate those better
than... than we do
now and communicate those better to.
Right.
And there's been this notion of thick data which includes the context
because frequently we've isolated the data from the actual context
and many of these things are very contextually bound.
And so do you see that there might be
a reinvestigation of the
where that data came from, what the context of the data was, etc.?
I think there might be a
a rise in methods that that
bring in the whole context,
bring in more of the context, multimodal inputs.
Do you know even even for
speech recognition it it's doing what it does
without even really knowing the domain that it's working in
and and that's pretty mindblowing really think
but when when it breaks down is when there are technical terms
when you're talking about a domain that is
less frequently talked about,
less represented and
bringing in that domain knowledge
I think is going to be huge.
And similarly in terms of, of
helping to create text alternatives for things,
the domain knowledge will will help to
give a better kind of base suggestion from the AI.
And perhaps with dialog
we can prompt people with the right questions to help them decide
is this is this actually a decorative image
or is it important for me to describe what's in this image?
You know, that's not actually always a trivial question to answer.
And of course, that brings in the issue of classification and labeling
and the need to box or classify specific things.
And many of these things are very fuzzy contexts
and classifiers are also determined hierarchically.
And there's yes.
So yeah, maybe we don't need a perfect classifier, but we need
a good dialog where the,
the, the system knows what questions to ask
to help the person decide.
Right.
And oh, and I just saw a message from
Carlos saying to end the discussions
and Carlos, I'm wondering can we fit in one more question.
I actually have to stop at the top of the hour. So.
Okay.
So we will have an opportunity to answer the questions
that people have submitted in the question and answer dialog.
And we have access to those. So
Shari will be able to respond
to some of these additional questions that have been asked
and and apologies that we went a little over time.
Carlos. Okay. Turn it back over to you.
Okay. Thank you. Thank you, Shari.
Thank You. Thank you
Thank you.
Shari and Jutta.
It was I was loving this discussion.
So it's really unfortunate that we we have stop now.
But thank you. Thank you so much for
your presentations.
Thank you all.
Thank you also to to all the panelists yesterday
and today for making this
a great symposium.
Lots of interesting and thought provoking ideas.
And thank you all for attending.