"Connection to storage server failed." - guam fails upon start



  • Hi folks!

    when I try to login to webmail I get an error: "Connection to storage server failed."

    I did poke around a bit: koladb.service seems fine, as is cyyrus-imapd.service. However, guam was dead and throws a lot of errors after restart:

    Jul 14 16:13:44 myFQDN guam[11039]: 16:13:43.994 [error] gen_fsm <0.1311.0> in state disconnected terminated with reason: no match of right hand value {{error,closed},true,<0.1311.0>,{[],[]}} in eimap:disconnected/2 line 137
    Jul 14 16:13:44 myFQDN guam[11039]: 16:13:43.995 [error] CRASH REPORT Process <0.1311.0> with 0 neighbours exited with reason: no match of right hand value {{error,closed},true,<0.1311.0>,{[],[]}} in eimap:disconnected/2 line 137 in gen_fsm:terminate/7 line 626

    Here is the last CRASH REPORT:

    2016-07-14 16:10:13 =ERROR REPORT====
    ** State machine <0.1225.0> terminating
    ** Last event in was {connect,<0.1184.0>,server_hello}
    ** When State == disconnected
    ** Data == {state,"127.0.0.1",9993,true,false,undefined,<<>>,1,{[],[]},undefined,undefined,false,undefined,<<>>,undefined,undefined,false}
    ** Reason for termination =
    ** {{badmatch,{{error,closed},true,<0.1225.0>,{[],[]}}},[{eimap,disconnected,2,[{file,"src/eimap.erl"},{line,137}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
    2016-07-14 16:10:13 =CRASH REPORT====
    crasher:
    initial call: eimap:init/1
    pid: <0.1225.0>
    registered_name: []
    exception exit: {{{badmatch,{{error,closed},true,<0.1225.0>,{[],[]}}},[{eimap,disconnected,2,[{file,"src/eimap.erl"},{line,137}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gen_fsm,terminate,7,[{file,"gen_fsm.erl"},{line,626}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
    ancestors: [<0.1184.0>,<0.93.0>,kolab_guam_sup,<0.88.0>]
    messages: []
    links: [<0.1184.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 3226
    neighbours:

    You can find more details (rpm list, guam config, logs, ...) here: http://pastebin.com/HJhtE5J2

    Does someone know what's wrong?

    Best regards,

    whitedwarf



  • Guam is failing to connect to the imap server. { error, closed } means the imap server has close the connection on it. Check the imap configuration parameters in the guam configuration file. In particular, check that cyrus-imap is listening on 9993 with implicit SSL (and that you are connecting to guam on 993 with implicit ssl)...



  • Thanks for the hint: I changed "tls" to "implicit_tls" for the imap_servers section of "kolab_guam" in /etc/guam/sys.config.
    It seems guam and cyrus can now talk to each other as I had to fix some permission issues with the certs and private key file cyrus is using.

    However the main problem is still there: "Connection to storage server failed." when I try to login.
    The crash report of guam now looks like:

    2016-07-19 16:15:33 =ERROR REPORT====
    "File operation error: emfile. Target: /opt/kolab_guam/lib/stdlib-2.8/ebin/sys.beam. Function: get_file. Process: code_server."
    2016-07-19 16:15:34 =ERROR REPORT====
    "File operation error: emfile. Target: sys.beam. Function: get_file. Process: code_server."
    2016-07-19 16:15:34 =CRASH REPORT====
    crasher:
    initial call: eimap:init/1
    pid: <0.2135.0>
    registered_name: []
    exception error: {undef,[{sys,print_log,[[]],[]},{gen_fsm,error_info,6,[{file,"gen_fsm.erl"},{line,654}]},{gen_fsm,terminate,7,[{file,"gen_fsm.erl"},{line,625}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
    ancestors: [<0.2113.0>,<0.90.0>,kolab_guam_sup,<0.88.0>]
    messages: []
    links: [<0.2113.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 610
    stack_size: 27
    reductions: 458
    neighbours:

    Best regards,

    whitedwarf



  • Using guam 0.8.2? If so, this is fixed in 0.8.3



  • Hi again,

    yes, I was using 0.8.2. I updated to guam-0.8.3-1.1.fc23.kolab_wf.x86_64. But there is no difference: the error message is exactly the same (with newer timestamps ;) )

    Best regards,

    whitedwarf



  • Hi!

    The error says a file is missing. So I poked around a bit:

    [root]# dnf -q whatprovides '*sys.beam'
    erlang-stdlib-18.3.3-1.2.fc23.kolab_wf.x86_64 : The Erlang standard libraries
    Repo : @System

    erlang-stdlib-18.3.3-1.2.fc23.kolab_wf.x86_64 : The Erlang standard libraries
    Repo : Kolab_Winterfell

    erlang-stdlib-17.4-5.fc23.x86_64 : The Erlang standard libraries
    Repo : fedora

    erlang-stdlib-17.4-7.fc23.x86_64 : The Erlang standard libraries
    Repo : updates

    [root]# rpm -ql erlang-stdlib-18.3.3-1.2.fc23.kolab_wf.x86_64 | grep sys.beam
    /usr/lib64/erlang/lib/stdlib-2.8/ebin/sys.beam

    Maybe this file can help?

    Best regards,

    whitedwarf

    Edit: A further look into the files revealed:

    [root]# rpm -qf /opt/kolab_guam/lib/stdlib-2.8/ebin/sys.beam
    erlang-stdlib-18.3.3-1.2.fc23.kolab_wf.x86_64
    [root]# ls -lah /opt/kolab_guam/lib/stdlib-2.8
    lrwxrwxrwx 1 root root 40 Jul 12 17:44 /opt/kolab_guam/lib/stdlib-2.8 -> ../../../usr/lib64/erlang/lib/stdlib-2.8

    Thus, that's not the root of all/my evil.



  • @whitedwarf said in "Connection to storage server failed." - guam fails upon start:

    The error says a file is missing.

    Which file / which error message?



  • The error message I posted earlier, in particular:

    2016-07-19 16:15:33 =ERROR REPORT====
    "File operation error: emfile. Target: /opt/kolab_guam/lib/stdlib-2.8/ebin/sys.beam. Function: get_file. Process: code_server."

    However, as I wrote before, there is a file /opt/kolab_guam/lib/stdlib-2.8/ebin/sys.beam because /opt/kolab_guam/lib/stdlib-2.8 is a symlink to /usr/lib64/erlang/lib/stdlib-2.8 and there is a ebin directory containing a file sys.beam (among others).



  • Hi!

    Any update on this one?

    Best regards,

    whitedwarf



  • Hi!

    I've updated to guam-0.8.3-1.2.fc23.kolab_wf.x86_64, still no change.

    I'd really like to fix this. Do you need more logs or can I help with other diag output?

    Best regards,

    whitedwarf


Log in to reply