Deadlock condition with script actor #34
Description
Found a new bug, a deadlock between the script actor and DSG sync lock mechanism. Happened when attempting to rez a script at the same time a sync message leading to a remove script was received. Below I posted two stack traces, and pointed the methods where that caused the deadlocks.
The main symptom is the Receiving loop of DSG hanging, waiting for m_Scripts lock, and the SyncOutUpdates locked, waiting for m_syncLock.
(Waiting for m_syncLock)
Not Flagged 6296 146 Worker Thread STP Util Thread #21 DSG.RegionSync.SyncInfoManager.UpdateSyncInfoByLocal Normal
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoManager.UpdateSyncInfoByLocal(OpenMetaverse.UUID uuid, System.Collections.Generic.HashSet<DSG.RegionSync.SyncableProperties.Type> updatedProperties) Line 128 + 0x20 bytes
DSG.RegionSync.dll!DSG.RegionSync.RegionSyncModule.OnSceneObjectPartUpdated(OpenSim.Region.Framework.Scenes.SceneObjectPart part, bool full) Line 2439 + 0x52 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.EventManager.TriggerSceneObjectPartUpdated(OpenSim.Region.Framework.Scenes.SceneObjectPart sop, bool full) Line 2986 + 0x1e bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPart.ScheduleFullUpdate() Line 2673 + 0x5d bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectGroup.ScheduleGroupForFullUpdate() Line 1890 + 0x24 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectGroup.aggregateScriptEvents() Line 1314 + 0xc bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPart.aggregateScriptEvents() Line 4719 + 0x29 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPart.SetScriptEvents(OpenMetaverse.UUID scriptid, int events) Line 3243 + 0xc bytes
OpenSim.Region.ScriptEngine.Shared.Instance.dll!OpenSim.Region.ScriptEngine.Shared.Instance.ScriptInstance.Load(System.AppDomain dom, string assembly, OpenSim.Region.ScriptEngine.Interfaces.StateSource stateSource) Line 344 + 0xd4 bytes
(Waiting for m_Scripts)
OpenSim.Region.ScriptEngine.XEngine.dll!OpenSim.Region.ScriptEngine.XEngine.XEngine.DoOnRezScript(object[] parms) Line 1316 + 0x8b bytes
OpenSim.Region.ScriptEngine.XEngine.dll!OpenSim.Region.ScriptEngine.XEngine.XEngine.OnRezScript(uint localID, OpenMetaverse.UUID itemID, string script, int startParam, bool postOnRez, string engine, int stateSource) Line 1014 + 0x10 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.EventManager.TriggerRezScript(uint localID, OpenMetaverse.UUID itemID, string script, int startParam, bool postOnRez, string engine, int stateSource) Line 1700 + 0x61 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPartInventory.CreateScriptInstance(OpenSim.Framework.TaskInventoryItem item, int startParam, bool postOnRez, string engine, int stateSource) Line 389 + 0x12c bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPartInventory.CreateScriptInstances(int startParam, bool postOnRez, string engine, int stateSource) Line 285 + 0x35 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectGroup.CreateScriptInstances(int startParam, bool postOnRez, string engine, int stateSource) Line 81 + 0xd6 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.Scene.RezObject(OpenSim.Region.Framework.Scenes.SceneObjectPart sourcePart, OpenSim.Framework.TaskInventoryItem item, OpenMetaverse.Vector3 pos, OpenMetaverse.Quaternion? rot, OpenMetaverse.Vector3 vel, int param) Line 2217 + 0x62 bytes
OpenSim.Region.ScriptEngine.Shared.Api.dll!OpenSim.Region.ScriptEngine.Shared.Api.LSL_Api.llRezAtRoot.AnonymousMethod__3(object x) Line 2973 + 0x2d0 bytes
OpenSim.Framework.dll!OpenSim.Framework.Util.FireAndForget.AnonymousMethod__1(object o) Line 1839 + 0x1f bytes
OpenSim.Framework.dll!OpenSim.Framework.Util.FireAndForget.AnonymousMethod__2(System.Threading.WaitCallback cb, object o) Line 1869 + 0x16 bytes
SmartThreadPool.dll!Amib.Threading.Internal.WorkItemsGroupBase.QueueWorkItem<System.Threading.WaitCallback,object>.AnonymousMethod__6(object state) Line 333 + 0x31 bytes
SmartThreadPool.dll!Amib.Threading.Internal.WorkItem.ExecuteWorkItem() Line 379 + 0x29 bytes
SmartThreadPool.dll!Amib.Threading.Internal.WorkItem.Execute() Line 312 + 0xa bytes
SmartThreadPool.dll!Amib.Threading.SmartThreadPool.ExecuteWorkItem(Amib.Threading.Internal.WorkItem workItem) Line 881 + 0x16 bytes
SmartThreadPool.dll!Amib.Threading.SmartThreadPool.ProcessQueuedItems() Line 821 + 0x10 bytes
[External Code]
Not Flagged 14408 34 Worker Thread SyncConnector0 (ReceiveLoop)
(Waiting for m_Scripts)
OpenSim.Region.ScriptEngine.XEngine.XEngine.OnRemoveScript Normal
OpenSim.Region.ScriptEngine.XEngine.dll!OpenSim.Region.ScriptEngine.XEngine.XEngine.OnRemoveScript(uint localID, OpenMetaverse.UUID itemID) Line 1380 + 0x29 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.EventManager.TriggerRemoveScript(uint localID, OpenMetaverse.UUID itemID) Line 1763 + 0x2f bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPartInventory.RemoveScriptInstance(OpenMetaverse.UUID itemId, bool sceneObjectBeingDeleted) Line 504 + 0xd0 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectPartInventory.RemoveScriptInstances(bool sceneObjectBeingDeleted) Line 321 + 0x54 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectGroup.RemoveScriptInstances(bool sceneObjectBeingDeleted) Line 99 + 0x86 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.Scene.DeleteSceneObject(OpenSim.Region.Framework.Scenes.SceneObjectGroup group, bool silent, bool removeScripts) Line 2359 + 0x1a bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.Scene.DeleteSceneObject(OpenSim.Region.Framework.Scenes.SceneObjectGroup group, bool silent) Line 2345 + 0x18 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.Scene.CrossPrimGroupIntoNewRegion(OpenMetaverse.Vector3 attemptedPosition, OpenSim.Region.Framework.Scenes.SceneObjectGroup grp, bool silent) Line 2446 + 0x16 bytes
OpenSim.Region.Framework.dll!OpenSim.Region.Framework.Scenes.SceneObjectGroup.AbsolutePosition.set(OpenMetaverse.Vector3 value) Line 461 + 0x5c bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoPrim.SetSOPAbsolutePosition(OpenSim.Region.Framework.Scenes.SceneObjectPart part, DSG.RegionSync.SyncedProperty pSyncInfo) Line 939 + 0xa4 bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoPrim.SetPropertyValue(OpenSim.Region.Framework.Scenes.SceneObjectPart part, DSG.RegionSync.SyncableProperties.Type property) Line 922 + 0x1a bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoPrim.SetPropertyValue(DSG.RegionSync.SyncableProperties.Type property) Line 567 + 0x14 bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoBase.UpdatePropertiesBySync(OpenMetaverse.UUID uuid, System.Collections.Generic.HashSet<DSG.RegionSync.SyncedProperty> syncedProperties) Line 215 + 0x17 bytes
(Waiting for m_syncLock)
DSG.RegionSync.dll!DSG.RegionSync.SyncInfoManager.UpdateSyncInfoBySync(OpenMetaverse.UUID uuid, System.Collections.Generic.HashSet<DSG.RegionSync.SyncedProperty> syncedProperties) Line 150 + 0x34 bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncMsgUpdatedProperties.HandleIn(DSG.RegionSync.RegionSyncModule pRegionContext) Line 751 + 0x7c bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncConnector.HandleMessage(DSG.RegionSync.SyncMsg msg) Line 426 + 0x22 bytes
DSG.RegionSync.dll!DSG.RegionSync.SyncConnector.ReceiveLoop() Line 400 + 0x10 bytes
[External Code]