Neil Walkinshaw

Associate Professor

Computer Science Building
Department of Informatics,
University of Leicester,
University Road,
Leicester,
LE1 7RH.

Tel: +44 (0)116 252 3953
Fax: +44 (0)116 252 3604
Email: nw91@le.ac.uk

Personal details

I am an associate professor in the Department of Informatics at the University of Leicester.

My research revolves around the development and application of Machine Learning and Data Analysis algorithms to established problems in the domain of Software Engineering and Quality Assurance.

I am currently also a visiting academic at the University of Sheffield.

Before coming to Leicester in 2010, I spent five years as a post-doctoral researcher with the Verification and Testing group in the Department of Computer Science at the University of Sheffield.

Before that I'd spent three years studying my PhD in the software engineering group (now SoSyg) in the department of Computer and Information Sciences at the University of Strathclyde in Glasgow.

Current program/scientific committees

  • The International Conference on Software Testing, 2018 (ICST'18)
  • The International Conference on Runtime Verification, 2017 (RV'17)
  • The International Workshop on Automation in Software Testing, Austin, Texas (AST 2017)

Administrative roles

Websites

Teaching

I am the convenor for courses on Software Systems Reengineering and Software Quality Assurance and Measurement.

I am in the extremely fortunate position to be lecturing in two modules that happen to be at the centre of my research interests:

  • Systems Re-engineering (CO7206) deals with the problem of understanding and maintaining legacy software systems.
  • Software Quality and Metrics (CO3095/7095) deals with the challenge of quality assessment and assurance.

These have both recently been made available as distance learning modules.

Publications

Please contact me for further information on any of these publications.

2017

O. Esoul, N. Walkinshaw
Extracting Packet Structures from Captured Network Data Using Segment-based Alignment
The International Conference on Quality, Security, and Reliability (QRS'17),
Prague, Czech Republic, July 2017

N. Walkinshaw, G. Fraser
Uncertainty-Driven Black-Box Test Data Generation
The International Conference on Software Testing (ICST'17)
Tokyo, Japan, March 2017

2016

N. Walkinshaw, M. Hall
Inferring Computational State Machine Models from Program Executions
The International Conference on Software Maintenance and Evolution (ICSME'16)
Raleigh, USA, October 2016

B. Eljuse, N. Walkinshaw
A Search-Based Approach for Stress-Testing Integrated Circuits
The International Symposium on Search Based Software Engineering (SSBSE'16)
Raleigh, USA, October 2016

R. Taylor, E. Tuosto, N. Walkinshaw, J. Derrick
Choreography-Based Analysis of Distributed Message Passing Programs
Formal Approaches to Parallel and Distributed Systems (4PAD),
24th Euromicro Conference on Parallel, Distributed, and Network-Based Processing (PDP'16)
Heraklion, Crete

2015

S. Nair, N. Walkinshaw, J. de la Vara, T. Kelly,
An Evidential Reasoning Approach for Assessing Confidence in Safety Evidence
The International Symposium on Software Reliability Engineering (ISSRE'15)
Gaithersburg, USA, November 2015

S. Scarle, N. Walkinshaw
Visualising Software as a Particle System
The International Working Conference on Software Visualisation (VISSOFT'15)
Bremen, Germany, 2015

T. Gransden, N. Walkinshaw, R. Raman
SEPIA: Search for Proofs using Inferred Automata
International Conference on Automated Deduction (CADE'15),
Berlin, Germany, 2015

G. Fraser, N. Walkinshaw
Assessing and Generating Test Sets in Terms of Behavioural Adequacy
Journal of Software Testing, Verification and Reliability (published online March 2015)

N. Walkinshaw and R. Taylor and J. Derrick
Inferring Extended Finite State Machine Models from Software Executions
Journal on Empirical Software Engineering (published online March 2015)

P. Papadopoulos, N. Walkinshaw
Black-Box Test Generation from Inferred Models
International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE'15)
Florence, Italy

2014

M. Hall, M. Khojaye, N. Walkinshaw, P. McMinn
Establishing the Source Code Disruption Caused by Automated Remodularisation Tools
The 30th International Conference on Software Maintenance and Evolution (ERA Track) - ICSME 2014
Victoria, Canada

S. Nair, N. Walkinshaw, T. Kelly
Quantifying Uncertainty in Safety Cases Using Evidential Reasoning
International Workshop on Next Generation of System Assurance Approaches for Safety Critical Systems (SASSUR 2014)
Florence, Italy

T. Gransden and N. Walkinshaw and R. Raman
Mining State Based Models from Proof Corpora
Mathematical Knowledge Management track
Conferences on Intelligent Computer Mathematics (CICM'14)
Coimbra, Portugal

2013

N. Walkinshaw and R. Taylor and J. Derrick
Inferring Extended Finite State Machine Models from Software Executions
The Working Conference on Reverse Engineering (WCRE'13)
Koblenz, Germany

N. Walkinshaw
Using Evidential Reasoning to Make Qualified Predictions of Software Quality
The 9th International Conference on Predictive Models in Software Engineering (PROMISE'13)
Baltimore, USA

N. Walkinshaw
Reverse Engineering Software Behaviour
Chapter - Elsevier Advances in Computers, volume 91, 2013

N. Walkinshaw, B. Lambeau, C. Damas, K. Bogdanov, P. Dupont
STAMINA: A Competition to Encourage the Development and Assessment of Software Model Inference Techniques
Journal of Empirical Software Engineering, 18(4), 2013

N. Walkinshaw, K. Bogdanov
Automated Comparison of State-Based Software Models in terms of their Language and Structure
ACM Transactions on Software Engineering and Methodology, 22(2) - 2013

2012

M. Hall, N. Walkinshaw, P. McMinn
Supervised Software Modularisation
The 2012 IEEE International Conference on Software Maintenance (ICSM 2012)

G. Fraser, N. Walkinshaw
Behaviourally Adequate Software Testing
Proceedings of the 2012 International Conference on Software Testing (ICST 2012)
Montreal, Canada

K. Meinke, N. Walkinshaw
Model-Based Testing and Model Inference
Proceedings of the International Symposium on Leveraging Applications of Formal Methods (ISOLA 2012), Heraklion, Crete

2011

N. Walkinshaw
Assessing Test Adequacy for Black-Box Systems Without Specifications
Proceedings of the 2011 23rd IFIP International Conference on Testing Software and Systems (ICTSS 2011)
Paris, France, November 2011

N. Walkinshaw, K. Bogdanov
Adapting Grammar Inference Techniques to Mine State Machines (Chapter),
Mining Software Specifications: Methodologies and Applications
D. Lo, S-C. Khoo, J. Han, and C. Liu (Eds),
Chapman Hall / CRC, Data Mining and Knowledge Discovery Series, 2011

S. Adra, M. Kiran, P. McMinn, N. Walkinshaw
A Multiobjective Optimisation Approach for Dynamic Inference and Refinement of Agent-Based Model Specifications
Proceedings of the 2011 IEEE Congress on Evolutionary Computation (CEC 2011)
New Orleans, USA, June 2011

2010

N. Walkinshaw, K. Bogdanov, J. Derrick, J. Paris
Increasing Functional Coverage by Inductive Testing: A Case Study
Proceedings of the 22nd IFIP International Conference on Testing Software and Systems (ICTSS 2010)
Natal, Brazil, November 2010

N. Walkinshaw
The Practical Assessment of Test Sets with Inductive Inference
Proceedings of Testing Academia & Industrial Conference (TAIC PART'10), 
Windsor, UK, August, 2010

N. Walkinshaw, K. Bogdanov, C. Damas, B. Lambeau, P. Dupont
A Framework for the Competitive Evaluation of Model Inference Techniques
Proceedings of the International Workshop on Model Inference in Testing (MIIT 2010)
Trento, Italy, July 2010

N. Walkinshaw, S. Afshan, P. McMinn
Using Compression Algorithms to Support the Comprehension of Program Traces
Proceedings of the International Workshop on Dynamic Analysis (WODA 2010)
Trento, Italy, July 2010

M. Hall, P. McMinn, N. Walkinshaw
Superstate Identification for State Machines Using Search-Based Clustering
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010)
Portland, USA, July 2010

S. Afshan, P. McMinn, N. Walkinshaw
Using Dictionary Compression Algorithms to Identify Phases in Program Traces
Technical Report CS-10-01
Department of Computer Science
The University of Sheffield, January, 2010

2009

K. Bogdanov, N. Walkinshaw
Computing the Structural Difference between State-Based Models
16th IEEE Working Conference on Reverse Engineering (WCRE 2009)
Lille, France, October 2009

N. Walkinshaw, J. Derrick
Incrementally Discovering Testable Specifications from Program Executions
Proceedings for Formal Methods for Components and Objects (FMCO'09), Springer LNCS(6286)
Eindhoven, Netherlands, November 2009

N. Walkinshaw, J. Derrick, Q. Guo
Iterative Refinement of Reverse-Engineered Models by Model-Based Testing
16th International Symposium on Formal Methods (FM 2009), Springer LNCS(5850)
Eindhoven, Netherlands, November 2009

Q. Guo, J. Derrick, N. Walkinshaw
Applying Testability Transformations to Achieve Structural Coverage of Erlang Programs
21st IFIP International Conference on Testing of Communicating Systems (TestCom 2009), Springer LNCS(5826)
Eindhoven, Netherlands, November 2009

2008

N. Walkinshaw, K. Bogdanov, M. Holcombe, S. Salahuddin
Improving Dynamic Software Analysis by Applying Grammar Inference Principles
Journal of Software Maintenance and Evolution, 20(4), pp.269-290, 2008

M.Kiran, S. Coakley, N. Walkinshaw, P. McMinn, M. Holcombe
Validation and Discovery from Computational Biology Model
BioSystems, 93(2008), pp. 141-150, 2008

N. Walkinshaw, K.Bogdanov, S. Ali, M. Holcombe
Automated Discovery of State Transitions and their Functions in Source Code
Software Testing, Verification and Reliability vol. 18, pp. 99-121, 2008

N. Walkinshaw, K.Bogdanov, K. Johnson
Evaluation and Comparison of Inferred Regular Grammars
9th International Colloquium on Grammar Inference (ICGI'08), Springer LNAI(5278)
St. Malo, France, September 2008

N. Walkinshaw, K.Bogdanov
Inferring Finite-State Models with Temporal Constraints
23rd IEEE/ACM International Conference on Automated Software Engineering (ASE'08)
L'Aquila, Italy, September 2008

2007

N. Walkinshaw, K.Bogdanov, M. Holcombe, S. Salahuddin
Reverse-Engineering State Machines by Interactive Grammar Inference
14th IEEE Working Conference on Reverse Engineering (WCRE'07)
Vancouver, Canada, October 2007

N. Walkinshaw, M. Roper, M. Wood
Feature Location and Extraction using Landmarks and Barriers
23rd IEEE International Conference on Software Maintenance (ICSM'07)
Paris, France, October 2007

F. Bernardini, M. Gheorghe, F. Romero-Campero, N. Walkinshaw
A Hybrid Approach to Modelling Biological Systems
8th Workshop on Membrane Computing, Springer Lecture Notes in Computer Science 4860/2007,
Thessaloniki, Greece, June 2007

N. Walkinshaw, K. Bogdanov
Applying Grammar Inference Principles to Dynamic Analysis
Program Comprehension and Dynamic Analysis (PCODA'07),
Vancouver, 2007

S. Ali, K. Bogdanov, N. Walkinshaw
A Comparative Study of Methods for Dynamic Reverse-Engineering of State Models
Technical Report CS-07-16 
Department of Computer Science
The University of Sheffield, October, 2007

2006

N. Walkinshaw, K. Bogdanov, M. Holcombe
Identifying State Transitions and their Functions in Source Code
1st IEEE Testing Academia & Industrial Conference (TAIC PART'06), 
Windsor, UK, August, 2006

Neil Walkinshaw
Partitioning Object-Oriented Software for Inspections
PhD. Thesis, The University of Strathclyde,
June 2006.

2005

N. Walkinshaw, M. Roper, M. Wood
Understanding Object-Oriented Source Code from the Behavioral Perspective
13th IEEE International Workshop on Program Comprehension (IWPC'05)
St. Louis, USA, May 2005

N. Walkinshaw, M. Roper, M. Wood
Collecting and Categorising Faults in Object-Oriented Code
UKTest 2005, 
Sheffield, September 2005

D. Kirk, M. Roper, N. Walkinshaw
Attribute Slicing
Dagstuhl Slicing Seminar Report
November 2005.

N. Walkinshaw, M. Roper and M. Wood
Extracting User-Level Functions from Object-Oriented Code
the ECOOP Workshop on Object-Oriented Reengineering (WOOR'05), 
Glasgow, July 2005

Serge Demeyer, Kim Mens, Roel Wuyts, Yann-Gaël Guéhéneuc, Andy Zaidman, Neil Walkinshaw, Ademar Aguiar, and Stéphane Ducasse
Report of the 6th international Workshop on Object-Oriented Reengineering
July 2005

N. Walkinshaw
Inspecting Object-Oriented Code from the Behavioural Perspective
Doctoral Symposium Abstract
ECOOP Doctoral Symposium, Glasgow, July 2005

2003

N. Walkinshaw, M. Roper, M. Wood
The Java System Dependence Graph
3rd IEEE International Workshop on Source Code Analysis and Manipulation (SCAM'03),
Amsterdam, Netherlands, September 2003

Research

My research revolves around the development and application of algorithms and associated tools to address established problems, predominantly in the domain of Software Engineering.

Specific challenges include:

  • the generation and assessment of test sets for black-box systems
  • the assessment of safety cases for safety-critical systems
  • the automatic formulation of higher-order logic proofs

My research interests span the areas of Software Engineering and Data Mining (and often focus on combining the two). I am currently focusing on developing techniques to solve the following problems:

  • Automated test generation for black-box components, such as Integrated Circuits or closed-source software components.
  • Reverse engineering behavioural models (such as state machines) from sequential data traces, such as network traces or function invocation sequences.
  • Aggregating uncertain evidence to facilitate complex, multifaceted decision problems. I am applying these techniques in Software Engineering to reason about evidence in safety arguments, and also to reason about uncertain decisions in autonomous systems.

I try to make all of my research software openly available.

Software

Some research software tools and prototypes that I have authored or contributed to:

Most of my research is applied; techniques are evaluated by developing prototypes for the sake of demonstration and assessment. This page lists some of these tools, which are all open source. If you are interested in contributing, or have any questions please contact me. Most of the tools are associated with a bug repository, so please report any bugs there.

The tools that use the GNU GPL only do so out of necessity (i.e. because one of the libraries is covered by GPL). Where possible, I try to use more permissive licences, such as BSD or Apache.

This page only lists the tools that I am actively working on at the moment (from 2013 onwards), where the work has been published. For tools from older papers, feel free to contact me.

The three tools that are under active development are:

  1. MINT - A tool to infer EFSMs from sequences with data (also used as the basis for ongoing work on software testing and automated theorem proving).
  2. SUMO - A tool for semi-automated remodularisation (and optionally refactoring) of software systems.
  3. ERTool - A tool to assist complex decisions (e.g. about software quality) in the presence of ignorance and uncertainty.

MINT - An Extended Finite State Machine Inference Tool

Extended Finite State Machines (EFSMs) are a modelling formalism that is frequently used to model software behaviour. They are effectively Finite State Machines with data guards. This tool will take traces, representing a sequence of events coupled with variable values, and infer the underlying EFSM.

The approach was described in "Inferring Extended Finite State Machines from Program Executions", which appeared at WCRE 2013. This has been elaborated upon in an extended Journal of Empirical Software Engineering paper.

The tool also underpins applications to automated interactive theorem proving, explored by Tom Gransden as part of his Ph.D. thesis. This work is described in two papers, one at CICM 2014, and a systems description paper at CADE 2015.

The tool takes as input a simple text file containing the sequences of events and variable values. The output can either be presented textually in the form of a GraphViz dot graph, or graphically in a GUI. The tool was developed with the help of Ramsay Taylor at the University of Sheffield, and Anselm Odigbo, who worked on the GUI.

The data constraints are inferred with the help of the WEKA data mining framework. Due to the use of this framework, the tool is licenced under the GPL v3 licence.

EFSMTool screenshot

SUMO - Supervised Software Remodularisation

As software evolves, its module structure can deteriorate. Given such a system, the task of rearranging its files into suitable modules can be very difficult. This tool employs a technique we call Supervised Remodularisation to, with help from the developer, rearrange a software system into suitable packages.

The approach was described in the paper Supervised Software Remodularisation at ICSM 2012, by Mat Hall, Neil Walkinshaw, and Phil McMinn. This has been followed up by an ICSME ERA paper in 2014 looking at the impact of re-modularisation on source code.

Mat Hall at the University of Sheffield is the principal developer of this tool.

The tool takes as input a SIL file (a textual file representing the current package structure of a system), and provides a GUI (supported by a constraint solver) to rearrange the modules. To give a rough idea, a screenshot of the GUI is pasted below. The tool allows the user to iteratively state which elements belong together (green edges), or should be kept apart (red edges), to gradually home-in on what they believe to be a suitable set of modules.

Recently, an Eclipse plugin has been developed, which enables the use of SUMO to refactor Java projects. This was implemented by Mohammad Ali Khojaye. The plugin is available here.

sumo screenshot

ERTool - Evidential Reasoning Tool

This tool implements a technique known as Evidential Reasoning, and was developed to investigate the application of Evidential Reasoning to assess software quality.

The technique was described in the paper Using Evidential Reasoning to Make Qualified Predictions of Software Quality at PROMISE'13. The approach has since been extended to reason about safety cases for safety-critical software (published at SASSUR'14).

Given a hierarchy of factors (e.g. quality factors such as usability, testability, code coverage, etc.), it enables the user to supply their subjective assessment in the lowest-level factors as a probability distribution. These are then propagated up the hierarchy to provide an overall assessment of the product.

The tool takes a small textual description of the tree as input, and produces output as a GraphViz dot graph, connecting the various distributions together, as follows:

promise example

PhysViz - Visualising Software as a Particle System

This investigates the use of games technology to visualise software systems. The core of the visualisation system is written by Simon Scarle at UWE. We developed a JSON format to represent software as networks, where nodes and edges are linked to metrics. Much of this was inspired by the work of the MOOSE team.

A paper on this was published at VISSOFT 2015.  More details on the software (including a link to the source code repository) are available here. As always, it is worth noting that this is a research prototype and could be difficult to work `out-of-the-box', but we are happy to help.

Funding

  • AWE NuSec Pilot project: Evidential Reasoning for Radiological Detection (2016)
  • DSTL ASUR phase-2: Trust for Autonomous Systems - Principal academic investigator, with Frazer Nash Consultancy and Qinetiq (2016)
  • InnovateUK TestMiner project (joint with DRTS) - Principal academic investigator (2016)
  • DfT TTRIG PREPARED project (Predicting, Preventing, and Analysing Rail Delays) - Principal Investigator (2015)
  • DSTL ASUR EVIRE project (An Evidence-Based Reasoning Framework to Support the Transparent Control, Verification, and Validation of Autonomous Systems) - Principal Investigator (2015)
  • DSTL HASTE project (joint with Critical Software and The University of Southampton) - PI for Leicester (2013-2014)
  • DSTL BATS project (Building Adequate Test Sets by Reverse Engineering) - Principal investigator (2012)
  • EPSRC REGI project (Reverse Engineering Software Models By Grammar Inference) (EP/F065825/1) - Recognised Researcher (employed as researcher 2009-2010)
  • EPSRC STAMINA project (EP/H002456/1) - CI / Recognised Researcher (2009-2012)
  • EU FP7 ProTest project - Research Assistant (2008-2009)
  • EPSRC AutoAbstract project - Research Assistant (finished 2008)

Supervision

I am currently supervising three PhD students.

  • Tom Gransden is investigating the use of Machine Learning algorithms to detect proof tactics (jointly supervised with Rajeev Raman). This research is funded by the College of Science and Engineering PhD Studentship Scheme
  • Othman Esoul is investigating the combination of Machine Learning algorithms with Fuzz-testing techniques to detect vulnerabilities in network protocols
  • Basil Eljuse (part time, working at ARM plc)  is investigating the application of search-based software testing techniques to testing the non-functional requirements of ARM ICs

I previously supervised Mohammad Al Shraa', who successfully gained his PhD in reverse-engineering design patterns in 2015.

Share this page: