1. Hey Guest, are you looking for a place to host your IRC related project? Then check out the Resource Manager on IRCForums! Click the resources tab above this notice!
  2. Hello Guest, are you new to IRCForums? Why not introduce yourself in the introduction forum!
  3. We have updated the layout of our forums. Please see this thread for more information.
Resources Introduce yourself! Updates

Is it possible to merge databases?

Discussion in 'Support' started by antoligy, Mar 4, 2012.

  1. antoligy
    Offline

    antoligy Newbie

    Member Since:
    Feb 28, 2012
    Message Count:
    4
    Likes Received:
    4
    Trophy Points:
    8
    I know that it's possible with Anope, but can two or more of Atheme's OpenSEX databases be merged?
    I've skimmed both the documentation and the repo -- but haven't found anything conclusive
    antoligy, Mar 4, 2012
    #1
  2. Bertrum
    Offline

    Bertrum ParadoxIRC

    Member Since:
    Dec 17, 2011
    Message Count:
    1,822
    Likes Received:
    243
    Trophy Points:
    188
    Yes, well, when I linked ParadoxIRC with AlphaChat last year, I converted my Anope database to Atheme. Then my new Atheme database was merged with the existing AlphaChat database. However matthew from AlphaChat completed the Atheme+Atheme database merge.

    So it is very possible, I'm just not 100% sure how matthew did it, however I'll refer him to this post and he may elaborate.
    Bertrum, Mar 4, 2012
    #2
    Medusa likes this.
  3. katsklaw
    Offline

    katsklaw Private

    Member Since:
    Dec 18, 2011
    Message Count:
    218
    Likes Received:
    90
    Trophy Points:
    58
    At least with Atheme 6, db's are plain text and copy and paste should work. HOWEVER, you'll want to resolve any duplicates first. As always, make backups before attempting anything.
    katsklaw, Mar 4, 2012
    #3
    Bertrum likes this.
  4. Bertrum
    Offline

    Bertrum ParadoxIRC

    Member Since:
    Dec 17, 2011
    Message Count:
    1,822
    Likes Received:
    243
    Trophy Points:
    188
    Oh the Atheme I was referring to was Atheme7 jsyk. Katsklaw indirectly reminded me :p
    Bertrum, Mar 4, 2012
    #4
  5. Trixar_za
    Offline

    Trixar_za GigIRC

    Member Since:
    Dec 18, 2011
    Message Count:
    1,295
    Likes Received:
    182
    Trophy Points:
    178
    There be an application for it in the contrib directory ;)
    (Pirate speak is fun ;))
    Trixar_za, Mar 4, 2012
    #5
  6. katsklaw
    Offline

    katsklaw Private

    Member Since:
    Dec 18, 2011
    Message Count:
    218
    Likes Received:
    90
    Trophy Points:
    58
    Perhaps you could be so kind as to offer a bit more info since there is more than 1 contrib directory and I haven't any file named anything like what I'd imagine a db merger tool would be named.
    katsklaw, Mar 4, 2012
    #6
  7. Trixar_za
    Offline

    Trixar_za GigIRC

    Member Since:
    Dec 18, 2011
    Message Count:
    1,295
    Likes Received:
    182
    Trophy Points:
    178
    Wait Wait. I'm thinking converting. There doesn't seem to be a script to combine two opensex dbs (which I think is kind of lame). There however IS a script in contrib named check_collisions.pl which you use to check for nickname and channel collisions between two dbs.
    Trixar_za, Mar 4, 2012
    #7
  8. nenolod
    Offline

    nenolod Atheme Moderator

    Member Since:
    Feb 20, 2012
    Message Count:
    58
    Likes Received:
    65
    Trophy Points:
    26
    Hmm, I could write a tool based on libathemecore if it would be useful.
    nenolod, Mar 5, 2012
    #8
  9. nenolod
    Offline

    nenolod Atheme Moderator

    Member Since:
    Feb 20, 2012
    Message Count:
    58
    Likes Received:
    65
    Trophy Points:
    26
    As a workaround, running check_collisions.pl and then running cat on the two databases to create a new database usually does the trick.
    nenolod, Mar 5, 2012
    #9
    Bertrum likes this.
  10. Bertrum
    Offline

    Bertrum ParadoxIRC

    Member Since:
    Dec 17, 2011
    Message Count:
    1,822
    Likes Received:
    243
    Trophy Points:
    188
    I think it would be helpful like :)
    Bertrum, Mar 5, 2012
    #10
  11. nenolod
    Offline

    nenolod Atheme Moderator

    Member Since:
    Feb 20, 2012
    Message Count:
    58
    Likes Received:
    65
    Trophy Points:
    26
    Hi,

    Atheme 7 now has a fsck-like tool called 'dbverify' which can be used for merging (as well as recovering from other errors that may result in a corrupted object store):


    Code (text):
    ciaranis:~/dev-src/atheme/src/dbverify$ cat ~/atheme/etc/services.db.fucko ~/atheme/etc/services.db > db.fucko
    ciaranis:~/dev-src/atheme/src/dbverify$ mv db.fucko ~/atheme/etc/services.db
    ciaranis:~/dev-src/atheme/src/dbverify$ ./dbverify
    [05/03/2012 16:01:39] dbverify is operating on services.db
    [05/03/2012 16:01:39] *** phase 1: demarshaling objects from opensex datastore
    [05/03/2012 16:01:39] *** phase 2: doing basic atheme database consistency check
    [05/03/2012 16:01:39] *** phase 3: verifying channel registration integrity
    [05/03/2012 16:01:39] *** phase 3: checking #moo for consistency
    [05/03/2012 16:01:39] *** phase 3: #moo: chanacs entry 'nenolod_' (0x644a24839e00) duplicates chanacs entry 0x644a24839e88
    [05/03/2012 16:01:39] *** phase 3: #moo: chanacs entry 'nenolod_' (0x644a24839d78) duplicates chanacs entry 0x644a24839e88
    [05/03/2012 16:01:39] *** phase 3: #moo: chanacs entry 'nenolod_' (0x644a24839cf0) duplicates chanacs entry 0x644a24839e88
    [05/03/2012 16:01:39] *** phase 3: #moo: chanacs entry 'nenolod_' (0x644a24839c68) duplicates chanacs entry 0x644a24839e88
    [05/03/2012 16:01:39] *** phase 4: verifying entity UID integrity
    [05/03/2012 16:01:39] *** phase 4: entity 'nenolod__' has duplicate EID 'AAAAAAAAE' (belonging to 'nenolod_'); regenerating as 'AAAAAAAAH'
    [05/03/2012 16:01:39] *** phase 4: 1 error(s) were found; running another pass
    [05/03/2012 16:01:39] *** phase 5: writing corrected state to object store
    This was an intentionally broken database, which dbverify fixed:

    Code (text):
    ciaranis:~/dev-src/atheme/src/dbverify$ ./dbverify
    [05/03/2012 16:04:19] dbverify is operating on services.db
    [05/03/2012 16:04:19] *** phase 1: demarshaling objects from opensex datastore
    [05/03/2012 16:04:19] *** phase 2: doing basic atheme database consistency check
    [05/03/2012 16:04:19] *** phase 3: verifying channel registration integrity
    [05/03/2012 16:04:19] *** phase 3: checking #moo for consistency
    [05/03/2012 16:04:19] *** phase 4: verifying entity UID integrity
    [05/03/2012 16:04:19] *** phase 5: writing corrected state to object store
    It will be part of beta1.
    nenolod, Mar 5, 2012
    #11
    antoligy, siniStar, katsklaw and 1 other person like this.
  12. Trixar_za
    Offline

    Trixar_za GigIRC

    Member Since:
    Dec 18, 2011
    Message Count:
    1,295
    Likes Received:
    182
    Trophy Points:
    178
    Oh I Love this - my db needs a fixin' ;)

    Also I could probably write a script that merges dbs in python if somebody is willing to provide me with two dbs :p
    Trixar_za, Mar 5, 2012
    #12
  13. nenolod
    Offline

    nenolod Atheme Moderator

    Member Since:
    Feb 20, 2012
    Message Count:
    58
    Likes Received:
    65
    Trophy Points:
    26
    A merge script is not really needed -- you can do it with dbverify just by catting the two together. It will reassign entity UIDs as needed and remove conflicts.
    nenolod, Mar 5, 2012
    #13
    Bertrum likes this.
  14. Trixar_za
    Offline

    Trixar_za GigIRC

    Member Since:
    Dec 18, 2011
    Message Count:
    1,295
    Likes Received:
    182
    Trophy Points:
    178
    So why not just make a bash script merges the two dbs using cat and then forwards it dbverify?
    Trixar_za, Mar 5, 2012
    #13
  15. nenolod
    Offline

    nenolod Atheme Moderator

    Member Since:
    Feb 20, 2012
    Message Count:
    58
    Likes Received:
    65
    Trophy Points:
    26
    I think the plan is to have a proper merge tool built into the import/export feature slated for 7.1.
    nenolod, Mar 5, 2012
    #14
  16. BlackDragon
    Offline

    BlackDragon Recruit

    Member Since:
    Jan 6, 2012
    Message Count:
    100
    Likes Received:
    11
    Trophy Points:
    23
    so... it is possible ?
    BlackDragon, Mar 29, 2013
    #15

Share This Page