1 code implementation • 25 Jul 2023 • Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Mark Harman
Existing research mostly improves the fairness of Machine Learning (ML) software regarding a single protected attribute at a time, but this is unrealistic given that many users have multiple protected attributes.
no code implementations • 14 Jul 2022 • Max Hort, Zhenpeng Chen, Jie M. Zhang, Mark Harman, Federica Sarro
How many datasets are used for evaluating bias mitigation methods?
2 code implementations • 7 Jul 2022 • Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Mark Harman
We find that (1) the bias mitigation methods significantly decrease ML performance in 53% of the studied scenarios (ranging between 42%~66% according to different ML performance metrics); (2) the bias mitigation methods significantly improve fairness measured by the 4 used metrics in 46% of all the scenarios (ranging between 24%~59% according to different fairness metrics); (3) the bias mitigation methods even lead to decrease in both fairness and ML performance in 25% of the scenarios; (4) the effectiveness of the bias mitigation methods depends on tasks, models, the choice of protected attributes, and the set of metrics used to assess fairness and ML performance; (5) there is no bias mitigation method that can achieve the best trade-off in all the scenarios.
1 code implementation • ICLR 2022 • Baptiste Roziere, Jie M. Zhang, Francois Charton, Mark Harman, Gabriel Synnaeve, Guillaume Lample
With little to no parallel data available for programming languages, unsupervised methods are well-suited to source code translation.
no code implementations • 8 Aug 2020 • Yixue Zhao, Justin Chen, Adriana Sejfia, Marcelo Schmitt Laser, Jie Zhang, Federica Sarro, Mark Harman, Nenad Medvidovic
UI testing is tedious and time-consuming due to the manual effort required.
Software Engineering
no code implementations • 15 Apr 2020 • John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Shan He, Ralf Lämmel, Erik Meijer, Silvia Sapora, Justin Spahr-Summers
Software-intensive organizations rely on large numbers of software assets of different types, e. g., source-code files, tables in the data warehouse, and software configurations.
no code implementations • 11 Apr 2020 • John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Ralf Lämmel, Erik Meijer, Silvia Sapora, Justin Spahr-Summers
We introduce the Web-Enabled Simulation (WES) research agenda, and describe FACEBOOK's WW system.
no code implementations • 19 Jun 2019 • Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu
This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research.
no code implementations • 24 May 2019 • Jie M. Zhang, Mark Harman, Benjamin Guedj, Earl T. Barr, John Shawe-Taylor
MV mutates training data labels, retrains the model against the mutated data, then uses the metamorphic relation that captures the consequent training performance changes to assess model fit.