fond
Model Checking Contest @ Petri Nets 2015
Bruxelles, Belgium, June 23, 2015
StrataGEM0.5.0 compared to other tools («Stripped» models, StateSpace)
Last Updated
August 19, 2015

Introduction

This page presents how StrataGEM0.5.0 do cope efficiently with the StateSpace examination face to the other participating tools. In this page, we consider «Stripped» models.

The next sections will show chart comparing performances in termsof both memory and execution time.The x-axis corresponds to the challenging tool where the y-axes represents StrataGEM0.5.0' performances. Thus, points below the diagonal of a chart denote comparisons favorables to the tool whileothers corresponds to situations where the challenging tool performs better.

You might also find plots out of the range that denote the case were at least one tool could not answer appropriately (error, time-out, could not compute or did not competed).

StrataGEM0.5.0 versus GreatSPN-Meddly

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for GreatSPN-Meddly, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to GreatSPN-Meddly are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 GreatSPN-Meddly Both tools   StrataGEM0.5.0 GreatSPN-Meddly
Computed OK 61 45 83   Smallest Memory Footprint
Do not compete 137 0 0 Times tool wins 69 120
Error detected 0 25 0   Shortest Execution Time
Cannot Compute + Time-out 48 176 75 Times tool wins 82 107


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus ITS-Tools

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for ITS-Tools, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to ITS-Tools are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 ITS-Tools Both tools   StrataGEM0.5.0 ITS-Tools
Computed OK 21 86 123   Smallest Memory Footprint
Do not compete 137 0 0 Times tool wins 41 189
Error detected 0 9 0   Shortest Execution Time
Cannot Compute + Time-out 35 98 88 Times tool wins 43 187


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus LTSMin

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for LTSMin, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to LTSMin are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 LTSMin Both tools   StrataGEM0.5.0 LTSMin
Computed OK 49 22 95   Smallest Memory Footprint
Do not compete 137 0 0 Times tool wins 110 56
Error detected 0 1 0   Shortest Execution Time
Cannot Compute + Time-out 22 185 101 Times tool wins 65 101


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus Marcie

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for Marcie, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to Marcie are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 Marcie Both tools   StrataGEM0.5.0 Marcie
Computed OK 32 56 112   Smallest Memory Footprint
Do not compete 137 0 0 Times tool wins 124 76
Error detected 0 4 0   Shortest Execution Time
Cannot Compute + Time-out 22 131 101 Times tool wins 55 145


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus pnmc

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for pnmc, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to pnmc are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 pnmc Both tools   StrataGEM0.5.0 pnmc
Computed OK 23 27 121   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 107 64
Error detected 0 0 0   Shortest Execution Time
Cannot Compute + Time-out 27 23 96 Times tool wins 38 133


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus PNXDD

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for PNXDD, so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to PNXDD are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 PNXDD Both tools   StrataGEM0.5.0 PNXDD
Computed OK 144 0 0   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 144 0
Error detected 0 0 0   Shortest Execution Time
Cannot Compute + Time-out 0 144 123 Times tool wins 144 0


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus TAPAAL(MC)

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for TAPAAL(MC), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to TAPAAL(MC) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 TAPAAL(MC) Both tools   StrataGEM0.5.0 TAPAAL(MC)
Computed OK 89 5 55   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 92 57
Error detected 0 0 0   Shortest Execution Time
Cannot Compute + Time-out 5 89 118 Times tool wins 99 50


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus TAPAAL(SEQ)

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for TAPAAL(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to TAPAAL(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 TAPAAL(SEQ) Both tools   StrataGEM0.5.0 TAPAAL(SEQ)
Computed OK 89 5 55   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 93 56
Error detected 0 0 0   Shortest Execution Time
Cannot Compute + Time-out 5 89 118 Times tool wins 100 49


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus TAPAAL-OTF(PAR)

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for TAPAAL-OTF(PAR), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to TAPAAL-OTF(PAR) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 TAPAAL-OTF(PAR) Both tools   StrataGEM0.5.0 TAPAAL-OTF(PAR)
Computed OK 89 8 55   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 103 49
Error detected 0 26 0   Shortest Execution Time
Cannot Compute + Time-out 23 78 100 Times tool wins 112 40


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart

StrataGEM0.5.0 versus TAPAAL-OTF(SEQ)

Some statistics are displayed below, based on 808 runs (404 for StrataGEM0.5.0 and 404 for TAPAAL-OTF(SEQ), so there are 404 plots on each of the two charts). Each execution was allowed 1 hour and 16 GByte of memory. Then performance charts comparing StrataGEM0.5.0 to TAPAAL-OTF(SEQ) are shown (you may click on one graph to enlarge it).

Statistics on the execution
  StrataGEM0.5.0 TAPAAL-OTF(SEQ) Both tools   StrataGEM0.5.0 TAPAAL-OTF(SEQ)
Computed OK 88 5 56   Smallest Memory Footprint
Do not compete 0 0 137 Times tool wins 97 52
Error detected 0 100 0   Shortest Execution Time
Cannot Compute + Time-out 69 52 54 Times tool wins 103 46


On the chart below, denote cases where the two tools did computed a result, denote the cases where at least one tool did not competed, denote the cases where at least one tool did a mistake and denote the cases where at least one tool stated it could not compute a result or timed-out.

memory chart time chart