So, uh, we have Willian Watanabe
from Universidade Tecnológica Federal do Paraná, in Brazil.
We have Yeliz Yesilada from the Middle East
Technical University, uh, Sheng Zhou
from Zhejiang University in China.
I hope I pronounced it correctly.
And Fabio Paternò from CNR
IST in Italy.
Okay, Thank you all for joining us. And
for some of you it’s earlier in the morning.
For others of you, it's later.
Well, for some of you, I guess it's really late in the evening.
So thank you all for your availability.
And let's start this discussion on how
I would say
current machine learning algorithms and current machine learning applications
can support or can improve methodologies for
automatically assessing web accessibility.
And from your previous works,
you’ve touched different aspects of how this can be done.
So machine learning has been used
to support web accessibility evaluation
through different aspects such as sampling, such as metrics,
such as evaluation predictions, such as handling dynamic pages.
And so and I understand that
these are domains, not all of these domains
you have work done on those, but some of you have worked on
specific domains.
And so I would like you to focus on the ones that you've been
working more closely.
And just for us to start,
just let us know what are the current challenges
that prevent further development and prevent further
use of machine learning or other A.I.
techniques in this specific domains?
Okay.
And I can start with you, Willian.
First of all, thank you very much for... for everything that is being organized,
it’s great to be here.
... Europe and this to give some context
and I'm Willian I'm a professor here in Brazil.
I work in accessibility,
my my focus, my research
focuses on web technologies, the ARIA specification
more specific and
just in regards to everything that has been said in the question
by Carlos Duarte, my focus is on evaluation prediction
according to the ARIA specification
and I believe the main...
I was invited to this...
to this panel
considering my research on identification of valences in web application.
So the problem that I address is
associated to identifying
components
In web applications. When we implement web applications, we use semi-structured
languages such as HTML.
My job is to identify what
these elements in the HTML structure represent
in the web page, like they can represent some
widgets, some specific type of widgets.
There's some components.
There are some landmarks that we need
to identify in the web page.
And this is basically what I do.
So what I have been doing for the last year,
I have been using machine learning for identifying these elements.
I use supervised learning and I use data
provided by the DOM structure of the web application.
So I search for elements in the web page and classifiy them as an element,
widgets or anything else.
The challenges in regards to that.
They are
are kind of different from the challenges
that have been addressed yesterday. Yesterday...
Yesterday... applications of machine learning.
I think they work with video in texts that are unstructured data.
So they are
more complicated, I would say.
And my... the main challenge that I that I address in my research
is associated with data acquisition and data extraction
where I identify what kind of features that I
I should use to identify these components in web applications
Associated with that I think they are and to summarize,
my problems are associated with the diversity of web applications.
There are different domains and
this kind of bias
and any dataset that we use, it's difficult.
For me. For instance, to identify,
a number of websites that implement
that represents all the themes of websites
that can be used, in web applications
variability in the implementation of HTML and JavaScript,
and the use of automatic tools for extracting this data
such as
web Driver API, the DOM structure dynamics and mutation observers.
There are a lot of specifications that are currently being developed
that I must use, and I always must
keep my observing to
to see if I can use them to improve my research.
And lastly, there is always the problem of manual classification in...
for generating these data sets that I can use
That’s it, Carlos.
Thank you.
Thank you Willian.
So Yeliz... and thank you Willian for introducing yourself
because I forgot to ask all of you that to do that.
So your first intervention, please
do give us a brief introduction about yourselves and the work you've been doing.
And so, Yeliz, I will follow with you.
Hi. Hello, everybody.
Good afternoon.
Afternoon for me.
So good afternoon, everybody.
I'm Yeliz.
I'm an associate professor at Middle East Technical University
Northern Cyprus Campus.
I've been doing web accessibility research for more than 20 years now.
Time goes really fast and recently I've been exploring machine learning
and AI specifically for web accessibility.
Supporting web accessibility from different dimensions.
Regarding the
challenges, I think there are of course many challenges.
But as Willian mentioned, I can actually say that
kind of the biggest challenge for my work has been data collection.
So I can actually
say that data, of course, is critical.
As it was discussed yesterday in the other panels,
Data is very critical for machine learning approaches
and for us collecting data,
making sure that the data is representing our user groups, different user groups,
and not biasing any user groups.
And also, of course, preparing
and labeling the data as certain
machine learning algorithms, of course, supervised ones they require labeling
and labeling has also been a challenge for us
because sometimes a certain task it's not so straightforward to do the labeling.
It's not black and white.
So it's been a challenge for us, I think in that sense.
And other two challenges I can mention is
I think the second one is the complexity of the domain.
When you think about the web accessibility, sometimes people think, Oh,
it's quite straightforward, but it's actually a very complex domain.
There are many different user groups, different user requirements,
so understanding those
and making sure that you actually address
different users and different requirements, it's quite challenging.
And since we also are working, this is the last one
that I wanted to mention, since we are also working with web pages.
They are complex, they are not well designed or well properly coded.
As we always say, browsers are tolerating, but for developing algorithms, machine
learning algorithms, they also have to deal
with those complexities, which makes the task quite complex.
I think.
So just to wrap up, I think in my work
there are three major challenges
data or the lack and quality of data.
Complexity of the domain, different users, different user
requirements and the complexity of the resources we are using.
So web pages, the source code and the complexity of
pages that are not
conforming to standards, I think they are really posing
a lot of challenges to algorithms that we are developing.
So these are all I wanted to say.
Okay, Thank you, Yeliz.
Very good
summary of major challenges
facing everyone that works in this in this field.
So thank you for that.
Sheng...
I want to go with you next. Okay.
Thank you, Carlos.
Hello everyone.
I'm Shen Zhou from Zhejiang University China
From my opinion view
I have three I think three challenges of course currently. Now.
First, I totally agree with the idea that it is
hard to acquire labels for more training.
Since the success of machine
learning heavily relies on a large number of labeled data,
however, accessing this data labels usually costs lots of time,
which may be hard to realize, especially in the accessibility domain.
I want to take the...
take the W4A...
Sorry, I'm a little bit nervous here, sorry...
I want to take the WCAG rule that's
we will want to take an image with text as an example.
As we discussed in the panel yesterday,
most of the current image captioning or OCR methods are trained on existing assets
rather than the image like logo that is essential in text alternative
The label for web accessibility evaluation
should fully consider the experience of different population.
There are very few datasets that are specifically designed
for the accessibility evaluation task and satisfies above requirements.
So the machine learning model is that traditional datasets cannot be
well generalized to accessibility evaluation.
Second, I think is about the web page sampling,
since I have done a little bit of work on this, I think
currently there are multiple factors that's affecting the sampling strategy.
First, sampling
has been a fundamental technique in web accessibility evaluation
when dealing with millions of pages. The previous page sampling
methods are usually based on the features of each page.
Such as the elements of the DOM tree structure.
The pages with similar features assumed to be generated by the same
development framework and have similar accessibility problems.
However, with the fast growth of web development framework
pages are developed with diverse tools.
For example, pages that looks very
similar may be developed by totally different framework and some pages
that look totally different may be developed by the same framework.
This poses great challenges for feature based Web Accessibility evaluation.
It is necessary to incorporate more factors
into the sampling process, such as the connection topology
among pages and a visual similarity and typesetting.
So how to identify the similarity between pages considering
multiple factors into a unified sampling probability
is critical for sampling.
I think this could be a problem that's related to the graph topology
content understanding
and metrical learning, which is a comprehensive research program.
So the third
challenge I think is the subjective evaluation rules.
When we evaluate the web accessibility,
there are both subjective and objective rules, right?
So for example, when evaluating the WCAG success
criterion, 1.4.5 images of text.
The image is expected to be associated with accurate
description of text which has been discussed in the panel yesterday.
It is still challenging to verify the matching
between the...
Yeah.
I guess, uh,
there are some connection issues.
Let's see. Okay.
He has dropped so.
So uh,
we’ll let Sheng... ok, he is coming back so
you're muted.
Oh, okay. All right. Okay. All right.
So can you.
Can you continue?
Okay. I'm so sorry.
Uh, okay. Okay.
I think there are three challenges.
And the first challenge is
same as Yeliz just described it.
That's we. It is harder to
we. You
dropped when you were starting to talk about the third challenge.
Okay.
Okay, So we still got the first and second challenge.
We, we heard that loud and clear.
So now you can resume on the third challenge.
Okay? Okay. Okay.
So the first challenge is, I think is the subjective evaluation rules.
This when evaluating the web accessibility
there are both subjective and objective rules.
For example, when evaluating the WCAG success criteria,
1.4.5 Images of text.
The image is expected to be associated with accurate
description text as discussed in the panel yesterday.
It is still challenging to verify whether the matching between image
with text, since we do not have access to the ground thruth of the
text of the image. So at
okay apparently
we lost.
Sheng again.
So let's just give him 10 seconds and see if he reconnects.
Otherwise we will move on to Fabio.
okay, so perhaps it's
better to to move on to Fabio and and
get the perspective of someone
who is making an automated accessibility evaluation tool available.
So it's certainly going to be interesting, so Fabio.
Can you can take it from here?
Yeah, yeah, yeah.
So, I’m Fabio, I’m a
Research director at the Italian National Research Council,
where I lead the laboratory on human interfaces and information systems, and we have
now a project funded by the National recovery and resilience
plan, which is about monitoring the
accessibility of the public administration websites.
And so, I mean, in this project we have our tool MAUVE++,
which is a tool open, freely available
and it has already more than 3000 registered users
and we recently performed an accessibility evaluation of
10,000 websites considering 200 pages for each website.
So it’s really large scale...
So we were very interested in understanding how machine learning
can help us
in these, you know, large scale monitoring work. So I mean, for this purpose...
I’m more research... so before this panel
I did a small, you know, systematic literature
review
So I went to the ACM digital library, I entered machine learning and accessibility evaluation
just curious to see what has been done so far.
So I got only 43 results which are not too many, I mean
I would have expected more. Then I looked through all these papers and actually
in the end, only 18 actually applied, because other papers were more
about, ok, machine learning can be interesting in future work, and so on.
I mean, so they say that the specific research efforts
have been so far limited
in this area, and another characteristic was that they were rather varied
in terms of the topic that they address.
So there are people who try to predict the website accessibility based on the accessibility of some pages
others try to check the meaningfulness of alternative descriptions
others classify user interface content elements.
So I would say that one challenge at this point is
well, machine
learning can give some, you know, useful complementary
support to the automatic tools
that we already have
as there are many... in theory there are more opportunities.
But then in practice there are a lot of problems.
Another challenge... identifying the relevant datasets and what are the features
that are really able to characterize the
type of aspects that we want to investigate.
And I would say the third and
last main general challenge is that we really
work with these computers who change. In the web
this means that how people implement, how people use
the application is continuously changing.
So there is also
the risk that the dataset becomes soon
obsolete, not sufficiently updated
for addressing all the emerging needs that can occur.
Okay.
Thank you for that perspective and Sheng
I want to give you now the opportunity
to finish up your intervention.
Okay.
Thank thank you, Carlos, and sorry for the lagging here
so and so I will continue my
third opening of the challenge. From my opinion
the third challenge is the subjective evaluation rules.
This one, evaluating web accessibility
there are both subjective and objective rules and one,
for example, when evaluating the image to text rule,
the image is expected to be associated with accurate description texts
and and and as discussed in the panel yesterday,
it is still challenging to verify the matching between the image
and the the text since there are no ground truth.
What kind of text should describe this image?
So as a result, the accessibility evaluation system is harder to justify
whether the alternate text really matches the image.
So, thanks.
Okay. Thank you.
And I'll take it from what I guess most of you.
Well, all of you have in one way or another mentioned one aspect of
web accessibility evaluation,
which is conformance to
the requirements to guidelines.
You, several of you mentioned the web content accessibility guidelines
in one way or another, and
checking what we do currently.
So far it's and following up on what Sheng
was just mentioning, are objective rules.
So that's what we can do so far, right?
Then when we start thinking about and because the guidelines are themselves
also subject to subjectivity and fortunately
at the
how can we try
to automate the access, the evaluation
of those more subjective guidelines or more subjective rules?
And how do you all think that artificial intelligence
algorithms or machine learning based approaches
can help us to assess conformance
to those technical requirements to to accessibility guidelines?
And I'll start with you now, Yeliz.
And thank you.
Carlos.
So regarding the conformance testing,
so maybe we can actually think of this as two kinds of problems.
The one is the testing, the other one is confirming basically repairing
or automatically fixing the problems.
So I see actually that
machine learning and AI in general
I think can help in both sides, in both parties.
So regarding the testing and auditing, if we take, for example,
WCAG evaluation methodology as the most systematic methodology
to evaluate for accessibility,
it includes, for example, five stages, five steps.
So I think
machine learning can actually help us in certain steps.
For example, it can help us to choose
a representative sample, which is the third step in WCAG-EM.
We are currently doing some work on that for example, to explore how to use
unsupervised learning algorithms to decide, for example,
what is a representative sample because Fabio, for example, mentioned
the problem of evaluating a large scale
websites with millions of pages.
So how do you decide for example, which ones to represent?
I mean, which ones to evaluate?
And do they really for example,
if you evaluate some of them,
how much of the sites you actually cover, for example.
So there I think machine learning and AI can help.
As I said, we are currently doing some work on that,
trying to explore machine learning algorithms for choosing
representative sample, making sure that the pages that you are
evaluating really represents the site and reduces the workload.
Because evaluating millions of pages, it's not an easy task.
So maybe we can pick certain sample pages and once we evaluate them,
we can transfer the knowledge from those pages
to the other ones because more or less pages
these days are developed with templates or automatically developed.
So maybe we can transfer the errors we identified
or the ways we are fixing to the others which are representative.
Regarding the step four in WCAG-EM...
That's actually about auditing the selected sample.
So how do you evaluate and test the sample?
I think in that part
as we all know, I mean Sheng mentioned there are a lot of ...
subjective rules which they require human testing.
So maybe there we need to explore more
how people, I mean how humans evaluate the certain
requirements
and how we can actually automate those processes.
So can we have machine learning algorithms that learn from how people
evaluate them, assess and implement those.
But of course, as we mentioned in the first part, data is critical
valid data and quality of data is very critical for those parts
regarding the repairing or automatically fixing certain problems.
I still I also think that machine learning algorithms can help.
For example, regarding
the images Sheng mentioned, we can automatically test
whether there is an alt text or not, but not the quality of the alt text.
So maybe there may be we can explore more and
do more about
understanding whether it's a good alt text or not
and try to fix it automatically by learning the
from the context and other aspects of the site.
Or I've been doing, for example, research in complex structures
like tables, they are also very difficult and challenging
for accessibility, for testing and for repairing.
We've been doing, for example, research in understanding
whether we can differentiate
and learn to differentiate a layout table from a data table.
And if it is a complex table, can we actually, for example, learn
how people are reading that and guiding the repairing of those?
We can, I guess, also do similar things with the forms
we can learn how people are interacting with forms
and try to some complex structures like forms or rich and dynamic content.
As Willian is working on.
So maybe we can actually do, for example, more work in there to automatically fix,
which can be encoded in, let's say, authoring tools or authoring environments
that they include AI without the developers
noticing that they are actually using AI to fix the problems.
So just to wrap up, I know I have a limited time
just to wrap up, so I see that ML can contribute in two things.
Both testing and repairing I think can help.
I agree and
some of the you things you mentioned are really
I guess they can be first steps.
We can assist a human expert,
the human evaluator, and take away some of the load.
And that's also what I, I take from from your intervention.
So, Fabio, I would like your your take on this.
I mean, actually
I think I agree with what Yeliz said before.
So first of all, we have to be aware of the complexity
of accessibility evaluation because we could just think about
WCAG 2.1, which is composed of 78 success
criteria, which are associated
with some hundreds of techniques,
of specific evaluation techniques. This is the kind of statement that it seems like
it is going to increase the number of techniques... and so on...
So the automatic support is really fundamental. And let’s say...
In general, when you use automatic support, the result over the check
would be okay, these are a pass... No, it fails
And the other one is cannot tell
So one possibility.
I think that can be interesting... how to exploit machine learning
in the situation which... you know... the automatic
solution is not able to deterministically provide
okay or fail. I mean, so these could be an interesting opportunity
which was also explored in the WADCHER European project.
So, in this case the idea was to allow
an accessibility validator
human accessibility expert
in this case to provide the input and then to try to use this input
in order to train the intelligent system
then it was not possible to extend it to
to validate these solutions. But,
for sure, for example, if I think about... it’s really easy automatically to detect
weather or not the alternative description exist.
It must much more difficult to say whether it is meaningful.
So, in this case, for example,
I have seen... also before it’s been mentioned... a lot of improvements in
AI applied to recognizing
in images what the content is.
So I have also seen that there's some attempt in this direction
has been performed, so we can think of situation in which
the AI take the image provides the descriptors
and then there is a kind of a similarity check, between these automatically
generated descriptions, the one that has been provided by the developer,
and see whether to some extent is meaningful. These, I think, is something
which is possible. What I’m not sure is how much we can find a general solution
so, a solution that can always work. I mean, so, I can see that this kind of AI
probably will be associated with some level of
confidence and then, I think, at this point we can also think of
leaving to the
end user decide what should be the level of confidence
that is acceptable when, you know, this automatic
support is used to understand the way that
the description, the alternative description, is meaningful.
So that would be the direction where I would
try, I mean, from the perspective of people who work on tools
for automatic validation and try to, you know, introduce
AI inside such
automatic frameworks. But another
key point that we have to be careful is the transparency.
I mean, when we talk about AI we often say
about the problem of the black box.
There is a lot of discussion about explainable AI. In explainable
AI, usually people try to say “oh the AI is not able to explain why this element
generated this result” or how can a change in this element, you know, obtained a different result.
What happens if a change
is handled this way.
So these, let’s say, questions in XAI
are also the questions that people encounter
when they have to interact with an evaluation tool.
And also, there is simply a study about the transparency of the tool.
So what about these tools that we have now available.
It was published
in ACM Transactions on Accessible Computing.
And it turned out that
even without AI, often these tools are a little bit black boxes.
They’re not sufficiently transparent, so,
for example, they say, we support this success criterion
but did not say which technique they actually apply for the purpose.
How these techniques are implemented.
So, let’s say, that often users are disoriented because
they use different tools they get different results
they do not understand the reason of such differences.
So let's say that this point of transparency is already
fundamental now that usually such validation tools do not use AI
we have to be careful that
if we add AI, should be added in such a way that is explainable
so that we can help people to better understand what happens in the evaluation
and not, you know, just giving results that we take as a
right without any sufficient explanation.
Yeah, I think that's a very important point
because if I'm a developer and I'm trying to solve
accessibility issues, I need to understand why is there an error...
not just that there is an error, over there.
So yeah, that's, that's a very important, very important point.
Thank you, Fabio.
So, Sheng, next to you.
Okay. Thanks.
And considering the incorporating
the artificial intelligence, I will try to find some way in
help the developers
so the first one is the code generation for automatically
fixing the accessibility problems. As Yilez just
said... web accessibility evaluation has been studied, but
we have to stand
at the view of the developers.
If the evaluation system only identify or locate
locate the accessibility problem,
it may be still hard for developers to fix these problems.
Things, some developers may lack experience on this,
and recently the artificial intelligence based code
generation has been well developed and given some
historical code on fixing accessibility problems
we have tried to train artificial intelligence model
to automatically detect the problem linked to a code snip
and to provide suggestions for the developers.
We expect that this function could help the developers fix
the accessibility problem and improve
their websites more efficiently.
And the second way is for the developers
is about the content generation.
As as discussed in the panel yesterday, there has been a
there have been several attempts in generating text alternates
for images or videos with the help of the computation of NLP
techniques.
However,
it may be not very practical for the image generators
to provide the text alternates since the state of the art methods usually requires
large models that are deployed on
GPU servers which is not...
which is not convenient for frequently updated images.
So recently we have been working
on some knowledge distillation methods, which aims at a
distill lightweight model from a large model
and we want to develop a lightweight artificial intelligence models
that can be deployed in the... browser extension or some lightweight
software.
We hope to reduce the time cost and the computation
cost of image providers and encourage them to conform
the accessibility technical requirements.
Okay. Thank you.
Thank you.
That's another very relevant points.
Make sure that whatever new techniques we develop
are really accessible to those who need to to use them.
And so the the computational resources are also
a very important aspect to take into account.
And so, Willian next your take on this,
please.
Okay. Okay.
Well, first, I would like to take from what Yeliz said that we
we have basically I it's nice to see that everyone is agreeing with everything
that has been said... is like we we talked before but we didn’t
we didn't talk at all
and so it's nice to see that everyone is having the same problems and
about what Yeliz said that she divided
the work of automatic evaluation in three steps.
The first one is testing and the second one is
automatically repairing accessibility on websites.
From my end and specifically,
I don't work with something that is,
I will say
subjective like image content generation.
I... my work mostly focused on identifying widgets.
And this is kind of objective, right?
It's a dropdown. It's not a toolkit.
This is something that I don't need to worry
about being sued over a bad classification or something else.
So... this is a different
aspect of accessibility that I work on and specifically my end
I work with supervised learning as everyone and...
classifying DOM elements as specific
components, interface components.
I, I use features extracted from the DOM structure. So
I think everyone
mentioned this, Sheng mentioned it as well.
Yeliz mentioned it in the question about tables and everything else and
I'm trying to use data
from websites that I evaluate as accessible
to enhance the accessibility of websites
that I don't... that don't implement these requirements.
For instance,
I see a website that implements rules, that implements the ARIA specification.
So I use it.
I extract data from it to to maybe apply it in a website
that doesn’t. This is kind of the, the work that I'm working,
this is kind of what I'm doing right now.
And... there is another thing.
So...
Fabio also mentioned the question about confidence.
I think this is kind of critical for us in terms of machine learning.
I think the word that we use usually is accuracy
and I believe that what will guide
each of us as researchers, whether we work on tests
or automatic repair, is basically the accuracy of our methodologies.
If I have
a lower
accuracy problem, I will use a testing approach.
Otherwise, I will try to automatically repair the webpage.
Of course, the best result we can get is automatic repair.
This is what will scale better for our users.
This is what will benefit more users
in terms of scale.
I think that it, Carlos. Everyone talked about everything that I wanted to say,
so this is mostly what I would say different.
So this is nice. Okay.
Still, let me just
a small provocation.
You said that you were
everything that you work in widget identification is objective.
I will disagree a little bit and I'm sure we can find several
examples of pages where you don't know if that's a link or a button.
It's so there can be subjectivity in there also.
So yeah, but just that, just a small provocation, as I say.
So we are fast approaching.
Yeah.
When
the conversation is good, time flies by so we are fast approaching the end.
So I will ask you to just quickly
comment on a final aspect, just one minute or two.
So please try to, to stick to that so that we don't go over time
and just you've already been in some ways
approaching this, but just what do you expect?
What would be one of the major contributions?
What are your future perspectives about the use of machine
learning techniques for web accessibility evaluation?
And I will start with you now, Fabio.
Okay, I mean, if I think
about a couple of interesting, you know, possibilities,
open up by machine learning, I mean,
you know.... when we.... when we have a user interface...
generally speaking we have two possible approaches.
So one is to look at the code, the associated generic interface
and see whether it is compliant with some rules. In other approaches
to look at how people interact with the system.
So to look at the logs of user interaction.
And so, in the past we did some work where we created a tool to identify
bad usability smells, which means
patterns of interaction that highlight there is some usability problems.
So for example, we look at mobile devices when there are a lot of pinch out, pinch in,
that means that probably the information is not well presented or
when people access continuously different links it means the links are too close, I mean...
so it's possible to identify sequences of interaction that highlight
there is a usability problem. So, one possibility, you know...
is to use some kind of machine learning for classifying
interaction with some assistive technology
that highlighted this kind of problems... that allow us from the data
to use experience that there are some specific
accessibility problems.
And... the second one... is about...
we mentioned before the importance of providing explanation
about a problem or why it is a problem and how to solve it.
So I think that would be the idea
in theory.... an idea application for a conversational agent.
Now there is a lot if discussion, for example, around ChatGPT
but
it’s very difficult, you know, to actually design
this case... a conversational agent that is able to take into account
the relevant context, which in this case is the type of user
that is actually now asking for help, because there are really many types of users
when people look at accessibility results, you know, that can be the web
commission with the person who has decided to have a service but don’t know anything
about its implementation, and it can be
the user, the disabled user, the developer, the accessibility expert.
Each of them require a different language, different terms, different
type of explanation, because when they look at... is this
website accessible, they really have different criteria
to understand the level of accessibility
and how to, then, operate in order to improve it.
So, this is one dimension of the complexity.
The other dimension of the complexity is the actual implementation.
It's really... we have... in this experience we are conducting in our laboratory
with these large scale validation.... ten thousand websites... it was really amazing
to see how different, you know, implementation languages... technical context...
people have used in order to
implement the website.
I mean, even people who have used the same
JavaScript frameworks, they can use them in very different ways
and so on.
So when you want to provide an explanation
often it’s disappointing just providing an understanding
a description of the errors... some standard examples
of how to solve the problem because often
there are different situations that require some specific
additional consideration for better explaining
how that error occurred, and what can be done in order to solve it.
But this part... this complexity... a good conversational agent for accessibility
would be a great result.
Thank you.
Sheng, you want to go next?
Okay so so time is limited.
I will save time.
I will talk about the future
perspective about the efficient page sampling.
According our data analyzed, we find that the page... the web pages
that with similar connection structure with other pages,
it usually have some similar accessibility problem.
So we tried to take this into...
take this into account for the accessibility evaluation.
And recently we used the graph neural networks,
which has been a hot research topic in machine learning community.
It combines both the network topology and the node, the attributes
and the unified representation for each node.
And here each node
Okay, I guess we lost Sheng again.
So in the interest of time I will skip immediately to you,
Willian.
Okay. See,
my take on this
I think it will be... pretty direct.
I, I think Fabio will talk about it, but we are all working
with specific guidelines inside of a set of guidelines
of accessibility guidelines, of WCAG.
And I think the the
the next step that we should address is associated with generalization
and incorporating into project ready projects into the project
that's just incorporated in any automatic evaluation tool.
And so in regards to all the problems
that we mentioned, associated to data acquisition, manual classification,
we had to find a way to scale our experiments
so that we can guarantee that it will work in any
theme or website.
I in regards to my research specifically, I think there are some I'm
trying to work in an automated generation of the structure for websites.
For instance, generating header structures and other
specific structures that the user can use
to practically... automatically enhance
the web accessibility of web pages
And I think I think that's it.
In regards to what you said, Carlos, just so that I can clear myself,
I... what I wanted to say is that... different from the panelists
from yesterday and different from Chaoai,
for instance, I think I'm working with
a simpler
machine learning approach.
I don't use deep learning, for instance, and since I don't see the
the use for it yet in my research
because my research
I think Yeliz mentioned that she she might use for labeling
and other stuff... like generation and I haven't reached that point yet.
I think there are some a lot of things that we can do with just with classification,
for instance.
That's it. Okay. Thank you.
And Yeliz, you want to conclude?
Yes, I actually
at least I hope that we will see developments again in two things.
I think the first one is automated testing.
I think we’re now at this stage that we have many tools and we know how
to implement and automate certain, for example, certain guidelines.
But there are a lot of bunch of others that they are very objective.
They require human evaluation.
It's very costly and expensive.
I think, from evaluation perspective.
So I'm hoping that there will be developments in machine learning
and AI algorithms to support and have more automation in those ones
that are really now requires the human
to do the evaluations.
And the other one is about the repairing.
So I'm also hoping that we will also see developments
in automating the kind of fixing the problems, automatically,
learning from the good examples and being able to develop solutions
while the pages are developed, they are actually automatically fixed
and sometimes may be seamless to the developers
so that they are not worried about the, you know, certain issues.
Of course, Explainability is very important to explain developers
what's going on, but I think automating certain things
there would really help automating the repairment.
Of course, to do that, I think we need datasets
and maybe hopefully in the community we will have shared datasets
that we can all work with and explore different algorithms.
As we know it's costly.
So exploring and doing research
with existing data, it helps a lot.
So I'm hoping that in the community we will see public datasets and of course
the technical skills are very important.
So human centered A.I., which is needed here I think is important.
So hopefully we will also see more people
contributing to that and the the development.
And of course, we should always remember, as Jutta
was mentioning yesterday, the bias is critical.
So when we are talking about, for example, automatically testing certain,
automating the test of certain rules, we should make sure that we are
not biasing certain user groups and we are really targeting everybody
and different user groups, different needs and users.
So that's all I wanted to say.
Thank you so much, Yeliz. And for bringing also that note to too.
I think it was a great way to finish this.
This panel.
So thank you so much to the four of you.
Really interesting to see all of those perspectives and what you
what you're working on and what you're planning
on doing so in the next
years.
I guess
let me just draw your attention.
There are several interesting questions on the Q&A.
So if you do have a chance, try to answer them there.
We unfortunately didn't have time to to get to those during our panel.
But I think there are and there are some that really have your names on it.
So you're exactly the
the correct person to answer those.
So once again, thank you so much for for your participation was great
and I will now have a shorter break
than the 10 minutes and has and will be back in 5 minutes.
So 5 minutes past the hour.