For those working on AIX, debugging or even understanding such cryptic error messages can be a painful time consuming task. I recently bumped into this error-
0514-516 Device Configuration Database Lock Service Timed Out
One the face of it, I knew that something related to the ODM (Object Data Manager) lock was going wrong, but I couldn’t exactly discern the problem. A quick google search on the phrase returned numerous results, most of them a verbatim copy of the standard AIX documentation which looks something like this-
It says, use the df command to find who’s holding up the ODM lock?! How can df command, whose job is reporting the free disk space available, tell you which process is holding up the ODM lock? I think there was a typo in the AIX docs and it trickled into all posts available online as a result of Ctrl-C,Ctrl-V.
The actual solution is to use the fuser command on the ODM lock file, which is a zero size file located at /etc/objrepos/config_lock and find out the PIDs of all processes which are holding/requesting the lock. And then grep for the PIDs in the ps -A output to find the processes names. When I did that I was indeed able to find the erring processes.
# fuser /etc/objrepos/config_lock
/etc/objrepos/config_lock: 8061020 8126500
# ps -A | grep 8061020
8061020 – 1:37 chdev
# ps -A | grep 8126500
8126500- 1:37 chginet
And on further checking the process stacks ( using proctsack command ) of the above 2 processes, I found that chdev was holding up the lock and then spawned chginet process which again wanted the same lock. Hence the timeout and the error.