Diff queries fail with out of memory error

[ZeLonewolf]

A complex diff query that I've been using successfully for some time is now failing. The failures happen on both the public instances as well as a private server that I run.

The query is below. It returns a list of certain types of streets which have changed in city 191230.

[timeout:1800][out:xml][diff:"2025-12-06T00:00:00Z"];

area(3600191230)->.city;

(
  way(area.city)[name][highway~"^(trunk|primary|secondary)$"][expressway=yes];
  way(area.city)[name][highway~"^(trunk|primary|secondary)$"][motorroad=yes];
  way(area.city)[ref][highway~"^(trunk|primary|secondary)$"][expressway=yes];
  way(area.city)[ref][highway~"^(trunk|primary|secondary)$"][motorroad=yes];
)->.expressways;

(
way(area.city)
  [name]
  [highway ~ "^(trunk|primary|secondary|tertiary|unclassified|residential|living_street|pedestrian|cycleway)$"]
  [expressway != yes][motorroad != yes];
way(area.city)
  [ref]
  [highway ~ "^(trunk|primary|secondary|tertiary|unclassified|residential|living_street|pedestrian|cycleway)$"]
  [expressway != yes][motorroad != yes];
);

(
  ._;
  way(area.city)[name][highway=service][foot=yes][bicycle=yes];
  way(area.city)[name][highway=service][service=alley];
  way[sidewalk~"^(yes|separate)$"].expressways;
  way["sidewalk:left"~"^(yes|separate)$"].expressways;
  way["sidewalk:right"~"^(yes|separate)$"].expressways;
  way["sidewalk:both"~"^(yes|separate)$"].expressways;
);

(
  ._; 
  //Remove trunk roads with no shoulder/sidewalk, unless explicitly tagged foot=yes
  - way[highway=trunk][sidewalk ~ "^(no|none)$"][shoulder = no][foot != yes]._;
);
(
  ._;
  //Remove private roads unless explicitly foot=yes
  - way[access ~ "^(private|customers|no)$"][foot!=yes]._;
);
(
  ._;
  //Remove areas
  - way[area = yes]._;
);
(
  ._;
  //Remove indoor
  - way[indoor = yes]._;
);
(
  ._;
  //Remove foot=no roads unless there's a sidewalk
  - way[foot ~ "^(no|private|discouraged)$"]
  [sidewalk!~"^(yes|separate)$"]
  ["sidewalk:left"!~"^(yes|separate)$"]
  ["sidewalk:right"!~"^(yes|separate)$"]
  ["sidewalk:both"!~"^(yes|separate)$"]._;
);
  
(._;>;);
out body;

[mmd-osm]

You could bump the default memory settings by adding [maxsize:2G] in the first line of your query. Try different maxsize values on your local instance first. The public instance is likely too busy for bigger memory limits.

[ZeLonewolf]

It will be over a week before I can test again. I did try setting larger maxsize values, and from my recollection I got immediate exception responses. Is there a server setting outside the query that's needed for larger maxsize values? Will report back once I have more data.

[mmd-osm]

The dispatcher has a total space limit across all queries. You can update it via the following command line parameter:

  --space=number: Set the memory limit for the total of all running processes to this value in bytes.

To check the current value, you need to run dispatcher --status
The current value is reported back as Total available space

[ZeLonewolf]

I've rebuilt my overpass server from scratch, on a fresh new (larger) hard drive. I am still getting this error.

I set the --space= value to 10GB

op         87891       1  3 23:08 pts/2    00:00:00 /opt/op/bin/dispatcher --osm-base --attic --allow-duplicate-queries=yes --space=107374182400 --db-dir=/opt/op/db
op         87903       1  0 23:08 pts/2    00:00:00 /opt/op/bin/dispatcher --areas --space=107374182400 --db-dir=/opt/op/db

This system has 32G of RAM and 120G of swap available.

I set [maxsize:2G] and got:

An error occurred during the execution of the overpass query! This is what overpass API returned:
runtime error: Query run out of memory using about 4838 MB of RAM.

This should be a small and pretty fast query that returns within a few seconds, as this is not a large city.

[ZeLonewolf]

I was finally able to get this to run by setting [maxsize:5G]. This seems excessively large for what this query does.