Extreme slowdowns with setRoomArea/resetRoomArea

[Nyyrazzilyss]

I'll need to adjust my cmud import script accordingly because of the changed setDoor command.

setDoor(4234, 'e', 1) - single quote or double quote btw?

I had adjusted the script previously, but i'm not sure if one of my compiles I ran was dev or 30 branch - I do know however that after exiting/restarting, it also deleted the re-written import script i'd previously done I'm going to guess that might have been the dev branch that did that, though i'm not sure. Would be unrelated to what you're doing right now, though i'll stick with branch 30 for now.

I think I did get your last branch 30 commit compiled however, and did see an error (which is likely related to me having to fix the setDoor command again)

"setDoor: bad argument #2 value (room with Id 48765 does not have a normal exit or a stub exit in direction "w".)"

setExit accepts (accepted) directions or numbers. setDoor is now direction only, not numbers?

(edit)

Or perhaps I just needed to look at it again - My code was already treating them differently. I'm recompiling the last commit right now.

[Nyyrazzilyss]

lua getRoomExits(48765)
{
south = 48763,
west = 48767,
east = 48766,
north = 48768
}

lua print ( setDoor(48765, "w", 2) )
nil setDoor: bad argument #2 value (room with Id 48765 does not have a normal exit or a stub exit in direction
"w".)
lua print ( setDoor(48765, 'w', 2) )
nil setDoor: bad argument #2 value (room with Id 48765 does not have a normal exit or a stub exit in direction
"w".)

lua print ( setDoor(48765, 'west', 2) )
nil setDoor: bad argument #2 value (room with Id 48765 does not have a special exit in direction "west".)
lua print ( setDoor(48765, "west", 2) )
nil setDoor: bad argument #2 value (room with Id 48765 does not have a special exit in direction "west".)

[Nyyrazzilyss]

I haven't imported the entire map yet, just tested a few lua commands with building doors on the new commit.

It is creating doors now - I might be a bit concerned though in that it's not consistent with getRoomExits - The room exit would be using the direction 'west', while the door is forcing it to use the direction 'w'.

I did exit/restart, and the door i'd created was still present.

Thanks!

lua getRoomExits(48765)
{
south = 48763,
west = 48767,
east = 48766,
north = 48768
}

lua getDoors(48765)
{}

lua print ( setDoor(48765, "w", 2) )
true

lua getDoors(48765)
{
w = 2
}

lua print ( setDoor(48765, "w", 0) )
true

lua getDoors(48765)
{}

lua print ( setDoor(48765, "west", 2) )
nil setDoor: bad argument #2 value (room with Id 48765 does not have a special exit in direction "west".)

[SlySven]

It might be possible to include a more generous setDoor(...) wrapper in ./src/mudlet-lua/lua/Other.lua that can take a wider range of exit directions - but I'm not in so much on the Lua part of the project (I've got my hands full with the C++ stuff!) that is kept in a separate GitHub repository owned by Vadim.

The only consistent thing I can say about exit direction keys for the different things that can be included on/as an exit is that: they are inconsistent!

[Nyyrazzilyss]

I'm working with map imports (from cmud) right now rebuilding my mudlet map files. At this time I expect it's that i've got more code in the script that needs to changed to the newer version of door handling - I'm running into issues with it locking up (with no errors provided/just frozen) mid import. I'm trying to decrease the import size to see if I can isolate when it's locking up, and find the exact command that causes it to occur. I -thought- I had everything changed

In refererence to rebuilding earlier btw - While there was problems with the -1 area earlier, my map builds were also completing at that time in 12 seconds. Now, it's minutes - Large difference in time. Was something being done now that's required that wasn't occuring earlier?

(edit)

Still testing to try and figure out where the crash/lockup was coming from. I've successfully had all maps rebuild individually. I'll try rebuilding at larger sizes again. I'm not sure if it might relate to the full map size being 30k rooms, and building individually was never more then 2k rooms at a time.

With the door change btw - Does that imply 3.0-e won't be backward compatible with 3.0-d? That might be something else to think about.

(edit 2)

Not having any luck getting it to consistently lockup. If I try importing a list of 40 areas, sometimes it completes, often it locks up. If I try using a subset of the same list, it might lockup at a random area (or complete). There's also no errors (and since it's using sql, nothing error/display/etc shows up on the screen until the script completes). Best guess? It's getting caught in an infinite loop inside the mudlet code. I'll change my script to use import one (the next) area every time I call it, and build the map myself piece by piece. On the plus side, i'm the only person that has to build the mapfiles (not the end user). Once i've forced it to rebuild all my maps, i'll continue with testing.

(edit 3)

I'm running an individual zone import right now (complete single zone, then repeat). Time to import each one is slowing down dramatically: It started at <1 sec, then has been increasing. The last one I did was 64/188, completed rooms are 8740/30,000. It took 28 seconds to complete the last area.

At 10436 rooms - 47sec for last area (291 rooms)

It might not be crashing at all, just taking longer then I want to wait, and i'm aborting it to quickly. When their were previous problems with the -1 area, it would complete through 30k rooms in 12 seconds. Something is definitly occuring with the -1 area that's really slowing it down. Does the function to find zone edges really need to be run on the -1 area (if that's what the problem is)? Note - I am only importing to -1, i'm not moving rooms to other areas

(edit 4)

It's likely very much my impatience, and not that it's crashing. I've been manually importing a zone every little while, and the last one was

zone: 87/188

Total room size now: 13068 ( out of expected 30,000 map size )

Last zone imported: 191 rooms

Time to complete (by stopwatch): 1min 18sec

I could probably (likely?) import the entire map again by setting an overnight import. That's not really helpful though, personally for example I made some major changes in my own map display with 12 second imports to be able to test.

I'd suggest not checking for zone exits on the -1 area probably what would be needed (hopefully!). It probably wouldn't be quite so noticable if I assigned real zones to rooms after creation, but I do keep ALL rooms in (-1) until my script decides that room has been 'found'.

In particular: If I add a room (-1) but don't assign a real area to it: Why would any area exists even be checked for?

[Nyyrazzilyss]

I've rewritten my import script a bit, and was able to change it from taking hours to minutes. It was rebuilding the exit map after every area import, instead of just once after all areas had been imported.

It did further isolate where the major slowdown was occuring, however (It's still not a matter of seconds to import)

setExit(FromID, ToID, dirtype) - majority of slowdowns
addRoom(roomid) - still slower then it should be

Over 31,000 rooms

addRoom - 6 minutes
setExit - 3 minutes

9 minutes in total combined - A big improvement.

I do think that comes back to -1 however - If FromID is inside (-1), zone edges should not be recalculated. When this code was somewhat broken and it wasn't doing anything on the (-1) area, that same combined action completed in 12 seconds.

Other then that on the import side, everything about it seems to be working properly now. I've been running with no problems combined with my mapper script. Also no longer seeing errors from the audit messages. It makes a huge difference with the usability of my mapper. +1, though you'll likely want at least one other person (with a different mapper script) to test.

[Nyyrazzilyss]

OK here's a slightly different one - The default area (Default Area) is showing up as first available map on the mapper/and selecting it does display the (-1) Area. Going with removing finding edges etc on (-1), it wouldn't hurt to not have Default Area as a selectable choice on the map. I'm not sure how that would impact other people/or other scripts.

[SlySven]

Well, the finding edges works both ways (entrances and exits are both monitored IIRC) - if a room is placing in a "normal" area but has an exit to the "default" one it still has to be tracked - it may be that the slow downs you are experiencing now is the fact the area -1 DOES NOW EXIST and it's area exits are now being tracked. Technically speaking the concept of the "Default" area was just that - the area a room goes to on first creation - Chris and I had different designs in this area and my choice was for an area with id 0 (which would automatically tend to be the value a new room's area value would get in the absence of an explicit initialisation to a different value - perhaps that is why Chris chose -1 so we could spot cases where a room hadn't been set up properly...?)

The whole reason for tracking area boundaries is that it is vital to operating the speedwalking/routefinding code - the A* search routine only works where the "heuristic" function (that estimates how far a possible room is from the destination so that the search can be optimised to "head in the right direction") requires that the x,y (and x) coordinates share a common reference which is not the case for rooms in a different area.

One thing you might consider is whether you could store the exit details for the rooms in the room user data and only create the exits when you move the rooms out of "storage" that should speed other things up because a) the isolated rooms will be completely eliminated from the speedwalk code and b) there will be less "edges" or routes between room to consider - which will be faster for the remaining rooms and also less exits to process overall until needed... this would also help with exits to rooms that are not currently in the loaded map - you will have them stored so your (specialised) setup can still refer to them - would that be worth considering?

As to hiding the default area (now it IS showing up) - that would be useful for some users and a pain for others. I will look to providing an option on the "Special Options" on the profile preference dialog to control this - which I will have default to on. For coding compatibility I think it will have to be common to ALL profiles but I can store it permanently (in the Mudlet 1.0.conf file) so it is remembered between runs without having to introduce a new parameter to store within the map or games save files.

[Nyyrazzilyss]

I guess a big part of this would have a lot to do with how I setup my map vs how it would impact maps from other muds.

For me, there's no way the user can enter the (-1) area - If they do, I move that room to the correct area (including creating it if needed) = They're not in the (-1) area.

I'm keeping everything in the (-1) area locked from speedwalks with lockRoom(). For my script, since all of that is 'unknown' to them, a generated speedwalk can never pass through the (-1) area. I am using (-1) as a holding area, and ideally the end user never even knows it exists. I'm not sure about other people, however, the unassigned area was not selectable on the map widget previously - Is it really necessary that it's going to be visible? In regards to an option to hide that which defaults to visible: That would work, as long as there was a command I could use in my script to force it to hidden.

I could move all the room exits into userdata, and not create the exits until the room is moved into the correct area. I don't think i'd want to do though though - It would speed up the time for me to build the map files, but by offloading potential slowdowns on to the end user.

I just doublechecked, and I do lock the room from speedwalks prior to creating its exits. The action of locking the room itself would probably have to generate the find edge request because something might have changed: But only if that room has exits in the first place. Since it doesn't have any existing exits yet though (I haven't built them yet)... lockRoom couldn't have effected edges.

It's likely my usage of the unassigned area in combination with lockRoom that's making it function (for me) as a holding area.

So how about the lockRoom request generates the find edges action, but -only- if that room has exits in the firstplace. Accompanying that, if a room is locked from speedwalk - Adding an exit to that room would -never- generate the request?

[SlySven]

I'll have to check but the following actions will cause the "rebuild the connection graph at the next opportunity" (actually the TMap::mMapGraphNeedsUpdate boolean) flag to be set:

• connectExitStub
• setRoomWeight
• lockRoom
• lockExit
• lockSpecialExit
• deleteArea
• setExit
• addRoom
• setRoomArea
• resetRoomArea
• setExitWeight
• addSpecialExit
• removeSpecialExit
• clearSpecialExits

HOWEVER the "graph" is only rebuilt when this flag is tested which is whenever the speedwalk/routefinding code is actually called (for) - which is why it takes a bit longer the first time it is used after one or more of these are done - there IS some debug output for this which debug builds of Mudlet will be producing...

OTOH: The area exit data is rebuilt when - ah, that is a bit harder to quantify but I think any-time an exit is changed - but it is not so easy to track through the code...

One thought - debug builds ARE slower (and about ten times larger) than release builds IIRC - which are you using?

EDIT: And the fact that the "Default" Area was not shown in the mapper was - I think - an incompletely documented (rather than undocumented) software feature which has been cleaned up.