fond
Model Checking Contest @ Petri Nets 2015
Bruxelles, Belgium, June 23, 2015
ITS-Tools compared to other tools («Stripped» models, ReachabilityFireabilitySimple)
Last Updated
August 19, 2015

Introduction

This page presents how ITS-Tools do cope efficiently with the ReachabilityFireabilitySimple examination face to the other participating tools. In this page, we consider «Stripped» models.

The next sections will show chart comparing performances in termsof both memory and execution time.The x-axis corresponds to the challenging tool where the y-axes represents ITS-Tools' performances. Thus, points below the diagonal of a chart denote comparisons favorables to the tool whileothers corresponds to situations where the challenging tool performs better.

You might also find plots out of the range that denote the case were at least one tool could not answer appropriately (error, time-out, could not compute or did not competed).

ITS-Tools versus Cunf

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for Cunf, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to Cunf are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools Cunf Both tools   ITS-Tools Cunf
Computed OK 161 15 60   Smallest Memory Footprint
Do not compete 0 277 0 Times tool wins 165 71
Error detected 7 1 0   Shortest Execution Time
Cannot Compute + Time-out 147 22 29 Times tool wins 166 70


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus GreatSPN-Meddly

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for GreatSPN-Meddly, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to GreatSPN-Meddly are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools GreatSPN-Meddly Both tools   ITS-Tools GreatSPN-Meddly
Computed OK 169 5 52   Smallest Memory Footprint
Do not compete 0 114 0 Times tool wins 184 42
Error detected 5 81 2   Shortest Execution Time
Cannot Compute + Time-out 107 81 69 Times tool wins 201 25


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus LoLA2.0

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for LoLA2.0, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to LoLA2.0 are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools LoLA2.0 Both tools   ITS-Tools LoLA2.0
Computed OK 46 83 175   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 48 256
Error detected 7 2 0   Shortest Execution Time
Cannot Compute + Time-out 169 0 7 Times tool wins 56 248


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus LTSMin

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for LTSMin, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to LTSMin are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools LTSMin Both tools   ITS-Tools LTSMin
Computed OK 45 83 176   Smallest Memory Footprint
Do not compete 0 0 0 Times tool wins 171 133
Error detected 4 3 3   Shortest Execution Time
Cannot Compute + Time-out 85 48 91 Times tool wins 157 147


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus Marcie

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for Marcie, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to Marcie are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools Marcie Both tools   ITS-Tools Marcie
Computed OK 79 25 142   Smallest Memory Footprint
Do not compete 0 0 0 Times tool wins 218 28
Error detected 7 0 0   Shortest Execution Time
Cannot Compute + Time-out 23 84 153 Times tool wins 196 50


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus TAPAAL(MC)

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for TAPAAL(MC), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to TAPAAL(MC) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools TAPAAL(MC) Both tools   ITS-Tools TAPAAL(MC)
Computed OK 51 68 170   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 90 199
Error detected 7 1 0   Shortest Execution Time
Cannot Compute + Time-out 154 6 22 Times tool wins 95 194


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus TAPAAL(SEQ)

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for TAPAAL(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to TAPAAL(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools TAPAAL(SEQ) Both tools   ITS-Tools TAPAAL(SEQ)
Computed OK 45 85 176   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 72 234
Error detected 7 0 0   Shortest Execution Time
Cannot Compute + Time-out 170 0 6 Times tool wins 101 205


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus TAPAAL-OTF(PAR)

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for TAPAAL-OTF(PAR), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to TAPAAL-OTF(PAR) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools TAPAAL-OTF(PAR) Both tools   ITS-Tools TAPAAL-OTF(PAR)
Computed OK 111 12 110   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 142 91
Error detected 7 7 0   Shortest Execution Time
Cannot Compute + Time-out 100 62 76 Times tool wins 141 92


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

ITS-Tools versus TAPAAL-OTF(SEQ)

Some statistics are displayed below, based on 808 runs (404 for ITS-Tools and 404 for TAPAAL-OTF(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing ITS-Tools to TAPAAL-OTF(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  ITS-Tools TAPAAL-OTF(SEQ) Both tools   ITS-Tools TAPAAL-OTF(SEQ)
Computed OK 97 24 124   Smallest Memory Footprint
Do not compete 0 137 0 Times tool wins 134 111
Error detected 7 0 0   Shortest Execution Time
Cannot Compute + Time-out 111 54 65 Times tool wins 134 111


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart