Skip to main content

crashtest.hxsmith.runner

Run many programs in parallel worker processes and summarise where they land on the ladder.

A run is a list of Jobs from any source: the random generator (fuzz), a corpus (crashtest.hxcorpus), or a file. execute_jobs is the one place that spawns workers, enforces the time limit, keeps artifacts and aggregates a Report.

class Report​

results: List[Result]​

results = field(default_factory=list)

valid: List[Result]​

invalid: List[Result]​

histogram​

def histogram() -> Dict[Quality, int]

clusters​

def clusters() -> List[Tuple[Quality, str, List[str]]]

Failures (below BEHAVES) grouped by signature, biggest first: the fix queue.

fix_order​

def fix_order() -> List[Tuple[str, int, int]]

Compile errors in the order to fix them: (kind, programs with it, programs it clears).

Greedy: each step takes the kind that makes the most programs compile-clean given the kinds already chosen (ties and dead ends fall back to the most frequent), so the first entries are the fixes that unblock the most programs, not just the most common message.

by_feature​

def by_feature() -> List[Tuple[str, int, float, float]]

Per feature: programs using it, their mean rung, and the share below BEHAVES. A feature with a low share-good is where the decompiler is weakest.

mean_quality​

def mean_quality() -> float

render​

def render(top: int = 12) -> str

from_json​

classmethod

@classmethod
def from_json(data: dict) -> 'Report'

to_json​

def to_json() -> dict

init​

def __init__(results: List[Result] = list()) -> None

execute_jobs​

def execute_jobs(makers: Sequence[Callable[[], Result]], jobs: int, out: Path, progress: Optional[Callable[[int, int, Result], None]] = None) -> Report

Run the callables in a thread pool (each drives a worker process) and write report.json.

fuzz​

def fuzz(seeds: Iterable[int], only: Optional[Iterable[str]] = None, exclude: Iterable[str] = (), jobs: int = 0, out: Optional[Path] = None, config: Optional[Config] = None, keep_above: Quality = Quality.BEHAVES, swarm: float = 0.0, progress: Optional[Callable[[int, int, Result], None]] = None) -> Tuple[Report, Path]

Generate a program per seed and run each.

keeper​

def keeper(keep_above: Quality) -> Callable[[Result], bool]

Keep artifacts of every program that did not reach keep_above, and of every invalid one.

lint​

def lint(seeds: Iterable[int], only: Optional[Iterable[str]] = None, exclude: Iterable[str] = (), jobs: int = 0, config: Optional[Config] = None, swarm: float = 0.0) -> List[Tuple[int, str]]

Check that the generator's programs are valid tests, without decompiling anything: each must compile, run, finish and repeat its own output. Returns (seed, problem) for the ones that do not. This is the loop for developing a feature.

lint_jobs​

def lint_jobs(items: Iterable[Job], jobs: int = 0) -> List[Tuple[str, str]]

lint for arbitrary jobs: (name, problem) for each that is not a valid test.

run_job​

def run_job(job: Job, out: Path, keep: Callable[[Result], bool], timeout: float = WORKER_TIMEOUT_S) -> Result

Evaluate one job in a worker. The artifact folder survives only if keep(result).

run_jobs​

def run_jobs(items: Iterable[Job], jobs: int = 0, out: Optional[Path] = None, keep_above: Quality = Quality.BEHAVES, progress: Optional[Callable[[int, int, Result], None]] = None) -> Tuple[Report, Path]

run_worker​

def run_worker(workdir: Path, timeout: float = WORKER_TIMEOUT_S) -> Optional[str]

Run the pipeline for the job in workdir in its own process group. Returns why it died, if it did.

RUNS​

RUNS = Path(__file__).resolve().parent.parent / 'hxsmith-runs'

WORKER_TIMEOUT_S​

WORKER_TIMEOUT_S = 180.0