http://wiki.docking.org/api.php?action=feedcontributions&feedformat=atom&user=Enkhjargal 2026-04-05T09:32:59Z User contributions MediaWiki 1.39.1 http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10479 2017-12-12T22:40:30Z

<p>Enkhjargal: </p> <hr /> <div>Look at the /nfs/db/trials/HOWTO.Enkhee<br /> <br /> <br /> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> zinc/SQL_statement/clinical_trial.sql<br /> <br /> Please see below the sql_statements<br /> <br /> 3. Load raw data to the database tables:<br /> If you want to delete all the existing data, then use --wipe<br /> Otherwise it will load data incrementally to the existing data.<br /> <br /> source /nfs/soft/www/apps/zinc15/envs/dev/bin/activate<br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_ct /nfs/db/trials/important/studies.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_condition /nfs/db/trials/important/browse_conditions.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_intervention /nfs/db/trials/important/interventions.txt<br /> <br /> <br /> ------------------------ zinc/SQL_statement/clinical_trial.sql ----------------------------------------<br /> <br /> 1. Creating the scheme and give the grants<br /> <br /> BEGIN;<br /> create schema clinical2;<br /> GRANT USAGE on schema clinical2 to root;<br /> GRANT USAGE on schema clinical2 to zincread;<br /> GRANT USAGE on schema clinical2 to zincfree;<br /> GRANT USAGE on schema clinical2 to test;<br /> GRANT USAGE on schema clinical2 to admin;<br /> GRANT USAGE on schema clinical2 to adminprivate;<br /> <br /> 2. Creating the tables and give grants<br /> <br /> BEGIN;<br /> create table clinical2.ctstatus (like clinical1.ctstatus including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctstatus FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO root;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO test;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO admin;<br /> <br /> create table clinical2.ctphase (like clinical1.ctphase including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctphase FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctphase TO root;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctphase TO test;<br /> GRANT ALL ON TABLE clinical2.ctphase TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctphase TO admin;<br /> <br /> create table clinical2.ct2 (like clinical1.ct2 including defaults including constraints including indexes);<br /> alter table clinical2.ct2 add constraint ct2_ctphase_fk_fkey foreign key (ctphase_fk) references clinical2.ctphase(ctphase_id);<br /> alter table clinical2.ct2 add constraint ct2_ctstatus_fk_fkey foreign key (ctstatus_fk) references clinical2.ctstatus(ctstatus_id);<br /> alter table clinical2.ct2 add column changed_date date;<br /> <br /> REVOKE ALL ON TABLE clinical2.ct2 FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2 TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2 TO test;<br /> GRANT ALL ON TABLE clinical2.ct2 TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2 TO admin;<br /> <br /> create table clinical2.ct2condclass (like clinical1.ct2condclass including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ct2condclass FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO admin;<br /> <br /> create table clinical2.ct2condition (like clinical1.ct2condition including defaults including constraints including indexes);<br /> alter table clinical2.ct2condition add constraint ct2condition_condclass_fk_fkey foreign key (condclass_fk) references clinical2.ct2condclass(ct2condclass_id);<br /> REVOKE ALL ON TABLE clinical2.ct2condition FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO admin;<br /> <br /> create table clinical2.ct2tocond (like clinical1.ct2tocond including defaults including constraints including indexes);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2condition_fk_fkey foreign key (ct2condition_fk) references clinical2.ct2condition(ct2condition_id);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2_fk_fkey foreign key (ct2_fk) references clinical2.ct2(ct2_id);<br /> REVOKE ALL ON TABLE clinical2.ct2tocond FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO test;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO admin;<br /> <br /> <br /> create table clinical2.ct2int (like clinical1.ct2int including defaults including constraints including indexes);<br /> alter tabel clinical2.ct2int drop column ct2_fk;<br /> REVOKE ALL ON TABLE clinical2.ct2int FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2int TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2int TO test;<br /> GRANT ALL ON TABLE clinical2.ct2int TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2int TO admin;<br /> <br /> <br /> create table clinical2.ct2subint (like clinical1.ct2subint including defaults including constraints including indexes);<br /> alter table clinical2.ct2subint add constraint ct2subint_ct2int_fk_fkey foreign key (ct2int_fk) references clinical2.ct2int(ct2int_id);<br /> alter table clinical2.ct2subint add constraint ct2subint_sub_id_fk_fkey foreign key (sub_id_fk) references substance(sub_id) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;<br /> REVOKE ALL ON TABLE clinical2.ct2subint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO admin;<br /> <br /> <br /> create table clinical2.ct2toint (ct2toint_id integer primary key serial not null,<br /> ct2_fk integer references clinical2.ct2,<br /> ct2int_fk integer references clinical2.ct2int)<br /> <br /> create sequence clinical2.ct2toint_ct2toint_seq;<br /> alter table clinical2.ct2toint alter column ct2toint_id set data type not null default nextval('clinical2.ct2toint_ct2toint_seq'::regclass);<br /> REVOKE ALL ON TABLE clinical2.ct2toint_ct2toint_seq FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO admin;<br /> <br /> <br /> REVOKE ALL ON TABLE clinical2.ct2toint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO admin;<br /> <br /> 3. Load data from existing tables<br /> <br /> insert into clinical2.ctstatus select * from clinical1.ctstatus;<br /> insert into clinical2.ctphase select * from clinical1.ctphase;<br /> insert into clinical2.ct2condclass select * from clinical1.ct2condclass;<br /> <br /> 4. Matching the name between subname.name and intervention.name<br /> <br /> create temp table tempsubname<br /> as select sub_id_fk as sub_id_fk, who_name as name<br /> from catalog_item join catalog on (cat_id_fk=cat_id and short_name='chembl20')<br /> join chembl20.molecule_dictionary as md on supplier_code=md.chembl_id<br /> join chembl20.molecule_atc_classification as mac on md.molregno=mac.molregno<br /> join chembl20.atc_classification as ac on mac.level5=ac.level5;<br /> <br /> insert into tempsubname<br /> select cs.sub_id_fk, sy.synonym<br /> from catalog_substance as cs join synonym as sy on cs.cat_content_fk = sy.cat_content_fk<br /> where not exists (select 1 from tempsubname as sn where sn.sub_id_fk = cs.sub_id_fk and sn.name = sy.synonym);<br /> <br /> alter table tempsubname add column q tsquery;<br /> <br /> update tempsubname as s<br /> set q=plainto_tsquery(t.name)<br /> from tempsubname as t<br /> where s.sub_id_fk=t.sub_id_fk and s.name=t.name;<br /> <br /> <br /> alter table clinical2.ct2int add column terms tsvector;<br /> update table clinical2.ct2int set term=to_tsvector('english', name)<br /> <br /> insert into clinical2.ct2subint (sub_id_fk, ct2int_fk)<br /> select distinct sub.sub_id_fk, int.ct2int_id<br /> from clinical2.ct2int as int join tempsubname as sub on<br /> int.terms@@sub.q<br /> where not exists (select 1 from clinical2.ct2int as int join tempsubname as sub on int.terms@@sub.q);<br /> <br /> <br /> 5. Warehousing<br /> <br /> update clinical2.ct2int as ct2int set num_trials = (select<br /> count(*) from clinical2.ct2toint as ct2toint where<br /> ct2int.ct2int_id = ct2toint.ct2int_fk);<br /> <br /> update clinical2.ct2int as ct2int set num_substances = (select<br /> count(*) from clinical2.ct2subint as ct2subint where<br /> ct2int.ct2int_id = ct2subint.ct2int_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_trials = (select<br /> count(*) from clinical2.ct2tocond as ct2tocond where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_substances = (select<br /> count(distinct(ct2subint.sub_id_fk)) from clinical2.ct2tocond as ct2tocond, clinical2.ct2toint as ct2toint, clinical2.ct2subint as ct2subint where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk and ct2tocond.ct2_fk = ct2toint.ct2_fk and ct2toint.ct2int_fk=ct2subint.ct2int_fk);</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10353 2017-09-28T17:34:41Z

<p>Enkhjargal: </p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> zinc/SQL_statement/clinical_trial.sql<br /> <br /> Please see below the sql_statements<br /> <br /> 3. Load raw data to the database tables:<br /> If you want to delete all the existing data, then use --wipe<br /> Otherwise it will load data incrementally to the existing data.<br /> <br /> source /nfs/soft/www/apps/zinc15/envs/dev/bin/activate<br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_ct /nfs/db/trials/important/studies.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_condition /nfs/db/trials/important/browse_conditions.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_intervention /nfs/db/trials/important/interventions.txt<br /> <br /> <br /> ------------------------ zinc/SQL_statement/clinical_trial.sql ----------------------------------------<br /> <br /> 1. Creating the scheme and give the grants<br /> <br /> BEGIN;<br /> create schema clinical2;<br /> GRANT USAGE on schema clinical2 to root;<br /> GRANT USAGE on schema clinical2 to zincread;<br /> GRANT USAGE on schema clinical2 to zincfree;<br /> GRANT USAGE on schema clinical2 to test;<br /> GRANT USAGE on schema clinical2 to admin;<br /> GRANT USAGE on schema clinical2 to adminprivate;<br /> <br /> 2. Creating the tables and give grants<br /> <br /> BEGIN;<br /> create table clinical2.ctstatus (like clinical1.ctstatus including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctstatus FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO root;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO test;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO admin;<br /> <br /> create table clinical2.ctphase (like clinical1.ctphase including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctphase FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctphase TO root;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctphase TO test;<br /> GRANT ALL ON TABLE clinical2.ctphase TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctphase TO admin;<br /> <br /> create table clinical2.ct2 (like clinical1.ct2 including defaults including constraints including indexes);<br /> alter table clinical2.ct2 add constraint ct2_ctphase_fk_fkey foreign key (ctphase_fk) references clinical2.ctphase(ctphase_id);<br /> alter table clinical2.ct2 add constraint ct2_ctstatus_fk_fkey foreign key (ctstatus_fk) references clinical2.ctstatus(ctstatus_id);<br /> alter table clinical2.ct2 add column changed_date date;<br /> <br /> REVOKE ALL ON TABLE clinical2.ct2 FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2 TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2 TO test;<br /> GRANT ALL ON TABLE clinical2.ct2 TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2 TO admin;<br /> <br /> create table clinical2.ct2condclass (like clinical1.ct2condclass including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ct2condclass FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO admin;<br /> <br /> create table clinical2.ct2condition (like clinical1.ct2condition including defaults including constraints including indexes);<br /> alter table clinical2.ct2condition add constraint ct2condition_condclass_fk_fkey foreign key (condclass_fk) references clinical2.ct2condclass(ct2condclass_id);<br /> REVOKE ALL ON TABLE clinical2.ct2condition FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO admin;<br /> <br /> create table clinical2.ct2tocond (like clinical1.ct2tocond including defaults including constraints including indexes);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2condition_fk_fkey foreign key (ct2condition_fk) references clinical2.ct2condition(ct2condition_id);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2_fk_fkey foreign key (ct2_fk) references clinical2.ct2(ct2_id);<br /> REVOKE ALL ON TABLE clinical2.ct2tocond FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO test;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO admin;<br /> <br /> <br /> create table clinical2.ct2int (like clinical1.ct2int including defaults including constraints including indexes);<br /> alter tabel clinical2.ct2int drop column ct2_fk;<br /> REVOKE ALL ON TABLE clinical2.ct2int FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2int TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2int TO test;<br /> GRANT ALL ON TABLE clinical2.ct2int TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2int TO admin;<br /> <br /> <br /> create table clinical2.ct2subint (like clinical1.ct2subint including defaults including constraints including indexes);<br /> alter table clinical2.ct2subint add constraint ct2subint_ct2int_fk_fkey foreign key (ct2int_fk) references clinical2.ct2int(ct2int_id);<br /> alter table clinical2.ct2subint add constraint ct2subint_sub_id_fk_fkey foreign key (sub_id_fk) references substance(sub_id) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;<br /> REVOKE ALL ON TABLE clinical2.ct2subint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO admin;<br /> <br /> <br /> create table clinical2.ct2toint (ct2toint_id integer primary key serial not null,<br /> ct2_fk integer references clinical2.ct2,<br /> ct2int_fk integer references clinical2.ct2int)<br /> <br /> create sequence clinical2.ct2toint_ct2toint_seq;<br /> alter table clinical2.ct2toint alter column ct2toint_id set data type not null default nextval('clinical2.ct2toint_ct2toint_seq'::regclass);<br /> REVOKE ALL ON TABLE clinical2.ct2toint_ct2toint_seq FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO admin;<br /> <br /> <br /> REVOKE ALL ON TABLE clinical2.ct2toint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO admin;<br /> <br /> 3. Load data from existing tables<br /> <br /> insert into clinical2.ctstatus select * from clinical1.ctstatus;<br /> insert into clinical2.ctphase select * from clinical1.ctphase;<br /> insert into clinical2.ct2condclass select * from clinical1.ct2condclass;<br /> <br /> 4. Matching the name between subname.name and intervention.name<br /> <br /> create temp table tempsubname<br /> as select sub_id_fk as sub_id_fk, who_name as name<br /> from catalog_item join catalog on (cat_id_fk=cat_id and short_name='chembl20')<br /> join chembl20.molecule_dictionary as md on supplier_code=md.chembl_id<br /> join chembl20.molecule_atc_classification as mac on md.molregno=mac.molregno<br /> join chembl20.atc_classification as ac on mac.level5=ac.level5;<br /> <br /> insert into tempsubname<br /> select cs.sub_id_fk, sy.synonym<br /> from catalog_substance as cs join synonym as sy on cs.cat_content_fk = sy.cat_content_fk<br /> where not exists (select 1 from tempsubname as sn where sn.sub_id_fk = cs.sub_id_fk and sn.name = sy.synonym);<br /> <br /> alter table tempsubname add column q tsquery;<br /> <br /> update tempsubname as s<br /> set q=plainto_tsquery(t.name)<br /> from tempsubname as t<br /> where s.sub_id_fk=t.sub_id_fk and s.name=t.name;<br /> <br /> <br /> alter table clinical2.ct2int add column terms tsvector;<br /> update table clinical2.ct2int set term=to_tsvector('english', name)<br /> <br /> insert into clinical2.ct2subint (sub_id_fk, ct2int_fk)<br /> select distinct sub.sub_id_fk, int.ct2int_id<br /> from clinical2.ct2int as int join tempsubname as sub on<br /> int.terms@@sub.q<br /> where not exists (select 1 from clinical2.ct2int as int join tempsubname as sub on int.terms@@sub.q);<br /> <br /> <br /> 5. Warehousing<br /> <br /> update clinical2.ct2int as ct2int set num_trials = (select<br /> count(*) from clinical2.ct2toint as ct2toint where<br /> ct2int.ct2int_id = ct2toint.ct2int_fk);<br /> <br /> update clinical2.ct2int as ct2int set num_substances = (select<br /> count(*) from clinical2.ct2subint as ct2subint where<br /> ct2int.ct2int_id = ct2subint.ct2int_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_trials = (select<br /> count(*) from clinical2.ct2tocond as ct2tocond where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_substances = (select<br /> count(distinct(ct2subint.sub_id_fk)) from clinical2.ct2tocond as ct2tocond, clinical2.ct2toint as ct2toint, clinical2.ct2subint as ct2subint where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk and ct2tocond.ct2_fk = ct2toint.ct2_fk and ct2toint.ct2int_fk=ct2subint.ct2int_fk);</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10284 2017-09-07T16:23:36Z

<p>Enkhjargal: </p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> zinc/SQL_statement/clinical_trial.sql<br /> <br /> Please see below the sql_statements<br /> <br /> 3. Load raw data to the database tables:<br /> If you want to delete all the existing data, then use --wipe<br /> Otherwise it will load data incrementally to the existing data.<br /> <br /> source /nfs/soft/www/apps/zinc15/envs/dev/bin/activate<br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_ct /nfs/db/trials/important/studies.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_condition /nfs/db/trials/important/browse_conditions.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_intervention /nfs/db/trials/important/interventions.txt<br /> <br /> <br /> ------------------------ zinc/SQL_statement/clinical_trial.sql ----------------------------------------<br /> <br /> 1. Creating the scheme and give the grants<br /> <br /> BEGIN;<br /> create schema clinical2;<br /> GRANT USAGE on schema clinical2 to root;<br /> GRANT USAGE on schema clinical2 to zincread;<br /> GRANT USAGE on schema clinical2 to zincfree;<br /> GRANT USAGE on schema clinical2 to test;<br /> GRANT USAGE on schema clinical2 to admin;<br /> GRANT USAGE on schema clinical2 to adminprivate;<br /> <br /> 2. Creating the tables and give grants<br /> <br /> BEGIN;<br /> create table clinical2.ctstatus (like clinical1.ctstatus including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctstatus FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO root;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO test;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO admin;<br /> <br /> create table clinical2.ctphase (like clinical1.ctphase including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctphase FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctphase TO root;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctphase TO test;<br /> GRANT ALL ON TABLE clinical2.ctphase TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctphase TO admin;<br /> <br /> create table clinical2.ct2 (like clinical1.ct2 including defaults including constraints including indexes);<br /> alter table clinical2.ct2 add constraint ct2_ctphase_fk_fkey foreign key (ctphase_fk) references clinical2.ctphase(ctphase_id);<br /> alter table clinical2.ct2 add constraint ct2_ctstatus_fk_fkey foreign key (ctstatus_fk) references clinical2.ctstatus(ctstatus_id);<br /> alter table clinical2.ct2 add column changed_date date;<br /> <br /> REVOKE ALL ON TABLE clinical2.ct2 FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2 TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2 TO test;<br /> GRANT ALL ON TABLE clinical2.ct2 TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2 TO admin;<br /> <br /> create table clinical2.ct2condclass (like clinical1.ct2condclass including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ct2condclass FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO admin;<br /> <br /> create table clinical2.ct2condition (like clinical1.ct2condition including defaults including constraints including indexes);<br /> alter table clinical2.ct2condition add constraint ct2condition_condclass_fk_fkey foreign key (condclass_fk) references clinical2.ct2condclass(ct2condclass_id);<br /> REVOKE ALL ON TABLE clinical2.ct2condition FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO admin;<br /> <br /> create table clinical2.ct2tocond (like clinical1.ct2tocond including defaults including constraints including indexes);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2condition_fk_fkey foreign key (ct2condition_fk) references clinical2.ct2condition(ct2condition_id);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2_fk_fkey foreign key (ct2_fk) references clinical2.ct2(ct2_id);<br /> REVOKE ALL ON TABLE clinical2.ct2tocond FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO test;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO admin;<br /> <br /> <br /> create table clinical2.ct2int (like clinical1.ct2int including defaults including constraints including indexes);<br /> alter tabel clinical2.ct2int drop column ct2_fk;<br /> REVOKE ALL ON TABLE clinical2.ct2int FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2int TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2int TO test;<br /> GRANT ALL ON TABLE clinical2.ct2int TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2int TO admin;<br /> <br /> <br /> create table clinical2.ct2subint (like clinical1.ct2subint including defaults including constraints including indexes);<br /> alter table clinical2.ct2subint add constraint ct2subint_ct2int_fk_fkey foreign key (ct2int_fk) references clinical2.ct2int(ct2int_id);<br /> alter table clinical2.ct2subint add constraint ct2subint_sub_id_fk_fkey foreign key (sub_id_fk) references substance(sub_id) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;<br /> REVOKE ALL ON TABLE clinical2.ct2subint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO admin;<br /> <br /> <br /> create table clinical2.ct2toint (ct2toint_id integer primary key serial not null,<br /> ct2_fk integer references clinical2.ct2,<br /> ct2int_fk integer references clinical2.ct2int)<br /> <br /> create sequence clinical2.ct2toint_ct2toint_seq;<br /> alter table clinical2.ct2toint alter column ct2toint_id set data type not null default nextval('clinical2.ct2toint_ct2toint_seq'::regclass);<br /> REVOKE ALL ON TABLE clinical2.ct2toint_ct2toint_seq FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO admin;<br /> <br /> <br /> REVOKE ALL ON TABLE clinical2.ct2toint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO admin;<br /> <br /> 3. Load data from existing tables<br /> <br /> insert into clinical2.ctstatus select * from clinical1.ctstatus;<br /> insert into clinical2.ctphase select * from clinical1.ctphase;<br /> insert into clinical2.ct2condclass select * from clinical1.ct2condclass;<br /> <br /> <br /> <br /> 4. Matching the name between subname.name and intervention.name<br /> <br /> create temp table subname<br /> as select sub_id_fk as sub_id_fk, who_name as name<br /> from catalog_item join catalog on (cat_id_fk=cat_id and short_name='chembl20')<br /> join chembl20.molecule_dictionary as md on supplier_code=md.chembl_id<br /> join chembl20.molecule_atc_classification as mac on md.molregno=mac.molregno<br /> join chembl20.atc_classification as ac on mac.level5=ac.level5;<br /> <br /> insert into subname<br /> select cs.sub_id_fk, sy.synonym<br /> from catalog_substance as cs join synonym as s on cs.cat_content_fk = s.cat_content_fk<br /> where not exists (select 1 from subname as sn where sn.sub_id_fk = cs.sub_id_fk and sn.name = sy.synonym);<br /> <br /> alter table subname add column q tsquery;<br /> <br /> update table subname as s<br /> set q=plainto_tsquery(t.who_name)<br /> from subname as t<br /> where s.sub_id_fk=t.sub_id_fk and s.who_name=t.who_name;<br /> <br /> alter table clinical2.ct2int add column terms tsvector;<br /> update table clinical2.ct2int set term=to_tsvector('english', name)<br /> <br /> <br /> insert into clinical2.ct2subint (sub_id_fk, ct2int_fk)<br /> select distinct sub.sub_id_fk, int.ct2int_id<br /> from clinical2.ct2int as int join subname as sub on<br /> int.terms@@sub.query;<br /> <br /> 5. Warehousing<br /> <br /> update clinical2.ct2int as ct2int set num_trials = (select<br /> count(*) from clinical2.ct2toint as ct2toint where<br /> ct2int.ct2int_id = ct2toint.ct2int_fk);<br /> <br /> update clinical2.ct2int as ct2int set num_substances = (select<br /> count(*) from clinical2.ct2subint as ct2subint where<br /> ct2int.ct2int_id = ct2subint.ct2int_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_trials = (select<br /> count(*) from clinical2.ct2tocond as ct2tocond where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk);<br /> <br /> update clinical2.ct2condition as ct2cond set num_substances = (select<br /> count(distinct(ct2subint.sub_id_fk)) from clinical2.ct2tocond as ct2tocond, clinical2.ct2toint as ct2toint, clinical2.ct2subint as ct2subint where<br /> ct2cond.ct2condition_id = ct2tocond.ct2condition_fk and ct2tocond.ct2_fk = ct2toint.ct2_fk and ct2toint.ct2int_fk=ct2subint.ct2int_fk);</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10274 2017-08-24T20:18:50Z

<p>Enkhjargal: </p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> zinc/SQL_statement/clinical_trial.sql<br /> <br /> Please see below the sql_statements<br /> <br /> 3. Load raw data to the database tables:<br /> If you want to delete all the existing data, then use --wipe<br /> Otherwise it will load data incrementally to the existing data.<br /> <br /> source /nfs/soft/www/apps/zinc15/envs/dev/bin/activate<br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_ct /nfs/db/trials/important/studies.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_condition /nfs/db/trials/important/browse_conditions.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_intervention /nfs/db/trials/important/interventions.txt<br /> <br /> <br /> ------------------------ zinc/SQL_statement/clinical_trial.sql ----------------------------------------<br /> <br /> 1. Creating the scheme and give the grants<br /> <br /> BEGIN;<br /> create schema clinical2;<br /> GRANT USAGE on schema clinical2 to root;<br /> GRANT USAGE on schema clinical2 to zincread;<br /> GRANT USAGE on schema clinical2 to zincfree;<br /> GRANT USAGE on schema clinical2 to test;<br /> GRANT USAGE on schema clinical2 to admin;<br /> GRANT USAGE on schema clinical2 to adminprivate;<br /> <br /> 2. Creating the tables and give grants<br /> <br /> BEGIN;<br /> create table clinical2.ctstatus (like clinical1.ctstatus including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctstatus FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO root;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctstatus TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO test;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctstatus TO admin;<br /> <br /> create table clinical2.ctphase (like clinical1.ctphase including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ctphase FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ctphase TO root;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ctphase TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ctphase TO test;<br /> GRANT ALL ON TABLE clinical2.ctphase TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ctphase TO admin;<br /> <br /> create table clinical2.ct2 (like clinical1.ct2 including defaults including constraints including indexes);<br /> alter table clinical2.ct2 add constraint ct2_ctphase_fk_fkey foreign key (ctphase_fk) references clinical2.ctphase(ctphase_id);<br /> alter table clinical2.ct2 add constraint ct2_ctstatus_fk_fkey foreign key (ctstatus_fk) references clinical2.ctstatus(ctstatus_id);<br /> alter table clinical2.ct2 add column changed_date date;<br /> <br /> REVOKE ALL ON TABLE clinical2.ct2 FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2 TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2 TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2 TO test;<br /> GRANT ALL ON TABLE clinical2.ct2 TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2 TO admin;<br /> <br /> create table clinical2.ct2condclass (like clinical1.ct2condclass including defaults including constraints including indexes);<br /> REVOKE ALL ON TABLE clinical2.ct2condclass FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condclass TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condclass TO admin;<br /> <br /> create table clinical2.ct2condition (like clinical1.ct2condition including defaults including constraints including indexes);<br /> alter table clinical2.ct2condition add constraint ct2condition_condclass_fk_fkey foreign key (condclass_fk) references clinical2.ct2condclass(ct2condclass_id);<br /> REVOKE ALL ON TABLE clinical2.ct2condition FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2condition TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO test;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2condition TO admin;<br /> <br /> create table clinical2.ct2tocond (like clinical1.ct2tocond including defaults including constraints including indexes);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2condition_fk_fkey foreign key (ct2condition_fk) references clinical2.ct2condition(ct2condition_id);<br /> alter table clinical2.ct2tocond add constraint ct2tocond_ct2_fk_fkey foreign key (ct2_fk) references clinical2.ct2(ct2_id);<br /> REVOKE ALL ON TABLE clinical2.ct2tocond FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2tocond TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO test;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2tocond TO admin;<br /> <br /> <br /> create table clinical2.ct2int (like clinical1.ct2int including defaults including constraints including indexes);<br /> alter tabel clinical2.ct2int drop column ct2_fk;<br /> REVOKE ALL ON TABLE clinical2.ct2int FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2int TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2int TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2int TO test;<br /> GRANT ALL ON TABLE clinical2.ct2int TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2int TO admin;<br /> <br /> <br /> create table clinical2.ct2subint (like clinical1.ct2subint including defaults including constraints including indexes);<br /> alter table clinical2.ct2subint add constraint ct2subint_ct2int_fk_fkey foreign key (ct2int_fk) references clinical2.ct2int(ct2int_id);<br /> alter table clinical2.ct2subint add constraint ct2subint_sub_id_fk_fkey foreign key (sub_id_fk) references substance(sub_id) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;<br /> REVOKE ALL ON TABLE clinical2.ct2subint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2subint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2subint TO admin;<br /> <br /> <br /> create table clinical2.ct2toint (ct2toint_id integer primary key serial not null,<br /> ct2_fk integer references clinical2.ct2,<br /> ct2int_fk integer references clinical2.ct2int)<br /> <br /> create sequence clinical2.ct2toint_ct2toint_seq;<br /> alter table clinical2.ct2toint alter column ct2toint_id set data type not null default nextval('clinical2.ct2toint_ct2toint_seq'::regclass);<br /> REVOKE ALL ON TABLE clinical2.ct2toint_ct2toint_seq FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint_ct2toint_seq TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint_ct2toint_seq TO admin;<br /> <br /> <br /> REVOKE ALL ON TABLE clinical2.ct2toint FROM PUBLIC;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO root;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincread;<br /> GRANT SELECT ON TABLE clinical2.ct2toint TO zincfree;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO test;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO adminprivate;<br /> GRANT ALL ON TABLE clinical2.ct2toint TO admin;<br /> <br /> 3. Load data from existing tables<br /> <br /> insert into clinical2.ctstatus select * from clinical1.ctstatus;<br /> insert into clinical2.ctphase select * from clinical1.ctphase;<br /> insert into clinical2.ct2condclass select * from clinical1.ct2condclass;<br /> <br /> <br /> <br /> 4. Matching the name between subname.name and intervention.name<br /> <br /> create temp table subname<br /> as select sub_id_fk as sub_id_fk, who_name as name<br /> from catalog_item join catalog on (cat_id_fk=cat_id and short_name='chembl20')<br /> join chembl20.molecule_dictionary as md on supplier_code=md.chembl_id<br /> join chembl20.molecule_atc_classification as mac on md.molregno=mac.molregno<br /> join chembl20.atc_classification as ac on mac.level5=ac.level5;<br /> <br /> insert into subname<br /> select cs.sub_id_fk, sy.synonym<br /> from catalog_substance as cs join synonym as s on cs.cat_content_fk = s.cat_content_fk<br /> where not exists (select 1 from subname as sn where sn.sub_id_fk = cs.sub_id_fk and sn.name = sy.synonym);<br /> <br /> alter table subname add column q tsquery;<br /> <br /> update table subname as s<br /> set q=plainto_tsquery(t.who_name)<br /> from subname as t<br /> where s.sub_id_fk=t.sub_id_fk and s.who_name=t.who_name;<br /> <br /> alter table clinical2.ct2int add column terms tsvector;<br /> update table clinical2.ct2int set term=to_tsvector('english', name)<br /> <br /> <br /> insert into clinical2.ct2subint (sub_id_fk, ct2int_fk)<br /> select distinct sub.sub_id_fk, int.ct2int_id<br /> from clinical2.ct2int as int join subname as sub on<br /> int.terms@@sub.query;</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10273 2017-08-24T19:58:29Z

<p>Enkhjargal: </p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> zinc/SQL_statement/clinical_trial.sql<br /> <br /> 3. Load raw data to the database tables:<br /> If you want to delete all the existing data, then use --wipe<br /> Otherwise it will load data incrementally to the existing data.<br /> <br /> source /nfs/soft/www/apps/zinc15/envs/dev/bin/activate<br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_ct /nfs/db/trials/important/studies.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_condition /nfs/db/trials/important/browse_conditions.txt <br /> python /nfs/soft/www/apps/zinc15/envs/dev/bin/zinc-manage utils clinical-trials load_intervention /nfs/db/trials/important/interventions.txt</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10272 2017-08-24T19:03:23Z

<p>Enkhjargal: </p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2. SQL queries to create a new clinical trial schema and table are located in zinc code:<br /> <br /> SQL_statement/clinical_trial<br /> <br /> 3. To load the data from raw data:</div>

Enkhjargal http://wiki.docking.org/index.php?title=Clinical_Trials_Loading&diff=10266 2017-08-17T19:04:07Z

<p>Enkhjargal: Created page with &quot; 1. Relevant Files: /nfs/db/trials/important --&gt; clinical trials raw data /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data http:/...&quot;</p> <hr /> <div> 1. Relevant Files:<br /> /nfs/db/trials/important --&gt; clinical trials raw data <br /> /nfs/home/teague/work/Projects/trials/extract.py --&gt; script to clean the raw data<br /> http://wiki.docking.org/index.php/Creating_clinical_name_mappings --&gt; how to create name_mappings<br /> <br /> 2.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10243 2017-08-09T21:07:14Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> screen -r sea16 on gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> sh SEAserver/scripts/stop-sea-server.sh<br /> sh SEAserver/scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> su - www<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> kill all sea-server related processes e.g. from htop<br /> make clean<br /> delete sea related libraries (fitcore, fpcore, libcore, seacore, seashell, seaserver) from your site-packages folder under the conda env<br /> rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver <br /> make all<br /> make SEAserver-start-production <br /> <br /> # run the test:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> # Redis config<br /> <br /> As for the Redis warnings, these are not new, but probably something worth taking care of (https://redis.io/topics/admin):<br /> <br /> add 'vm.overcommit_memory = 1' to /etc/sysctl.conf<br /> <br /> From here: The Linux kernel will always overcommit memory, and never check if enough memory is available. This increases the risk of out-of-memory situations, but also improves memory-intensive workloads.<br /> <br /> run echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> add 'echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled' to /etc/rc.local<br /> <br /> From here: Latency induced by transparent huge pages<br /> <br /> Unfortunately when a Linux kernel has transparent huge pages enabled, Redis incurs to a big latency penalty after the fork call is used in order to persist on disk. Huge pages are the cause of the following issue:<br /> <br /> Fork is called, two processes with shared huge pages are created.<br /> In a busy instance, a few event loops runs will cause commands to target a few thousand of pages, causing the copy on write of almost the whole process memory.<br /> This will result in big latency and big memory usage.<br /> <br /> Make sure to disable transparent huge pages using the following command:<br /> <br /> echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10219 2017-07-24T18:47:55Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> <br /> 2. In the flask code, change the database url to the newly created db <br /> <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> <br /> 4. Load the openfda regulatory_status data (populates 'Status' table in db, last updated April/2017): <br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> <br /> 5. Load the openfda excipients function data (populates 'Function' table in db, last updated April/2017):<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> <br /> 6. Load the dye relation data. (creates the relation for excipients to functions which are dye):<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> <br /> 7. Load the fda general additives data.(creates all other function and status relations to excipients): <br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> <br /> 8. Load the openfda drug label. (populate brand and substance tables and their relations to existing excipients):<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> <br /> 9. Pull Zincids for populated Substance data. (populate the zincid and smiles column in substance table)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data<br /> <br /> DB should be now fully populated.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10198 2017-07-06T17:45:58Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> screen -r sea16 on gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> sh SEAserver/scripts/stop-sea-server.sh<br /> sh SEAserver/scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> make clean<br /> delete sea related libraries (fitcore, fpcore, libcore, seacore, seashell, seaserver) from your site-packages folder under the conda env<br /> rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver <br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start-production cd SEAserver<br /> <br /> # run the test:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> # Redis config<br /> <br /> As for the Redis warnings, these are not new, but probably something worth taking care of (https://redis.io/topics/admin):<br /> <br /> add 'vm.overcommit_memory = 1' to /etc/sysctl.conf<br /> <br /> From here: The Linux kernel will always overcommit memory, and never check if enough memory is available. This increases the risk of out-of-memory situations, but also improves memory-intensive workloads.<br /> <br /> run echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> add 'echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled' to /etc/rc.local<br /> <br /> From here: Latency induced by transparent huge pages<br /> <br /> Unfortunately when a Linux kernel has transparent huge pages enabled, Redis incurs to a big latency penalty after the fork call is used in order to persist on disk. Huge pages are the cause of the following issue:<br /> <br /> Fork is called, two processes with shared huge pages are created.<br /> In a busy instance, a few event loops runs will cause commands to target a few thousand of pages, causing the copy on write of almost the whole process memory.<br /> This will result in big latency and big memory usage.<br /> <br /> Make sure to disable transparent huge pages using the following command:<br /> <br /> echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10120 2017-05-30T18:51:28Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> screen -r sea16 on gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> cd SEAsever<br /> sh scripts/stop-sea-server.sh<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> make clean<br /> delete sea related libraries (fitcore, fpcore, libcore, seacore, seashell, seaserver) from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> <br /> <br /> # run the test:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> # Redis config<br /> <br /> As for the Redis warnings, these are not new, but probably something worth taking care of (https://redis.io/topics/admin):<br /> <br /> add 'vm.overcommit_memory = 1' to /etc/sysctl.conf<br /> <br /> From here: The Linux kernel will always overcommit memory, and never check if enough memory is available. This increases the risk of out-of-memory situations, but also improves memory-intensive workloads.<br /> <br /> run echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> add 'echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled' to /etc/rc.local<br /> <br /> From here: Latency induced by transparent huge pages<br /> <br /> Unfortunately when a Linux kernel has transparent huge pages enabled, Redis incurs to a big latency penalty after the fork call is used in order to persist on disk. Huge pages are the cause of the following issue:<br /> <br /> Fork is called, two processes with shared huge pages are created.<br /> In a busy instance, a few event loops runs will cause commands to target a few thousand of pages, causing the copy on write of almost the whole process memory.<br /> This will result in big latency and big memory usage.<br /> <br /> Make sure to disable transparent huge pages using the following command:<br /> <br /> echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10119 2017-05-30T18:51:08Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> screen -r sea16 on gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> cd SEAsever<br /> sh scripts/stop-sea-server.sh<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> make clean<br /> delete sea related libraries (fitcore, fpcore, libcore, seacore, seashell, seaserver) from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> <br /> <br /> # run the test:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> # <br /> As for the Redis warnings, these are not new, but probably something worth taking care of (https://redis.io/topics/admin):<br /> <br /> add 'vm.overcommit_memory = 1' to /etc/sysctl.conf<br /> <br /> From here: The Linux kernel will always overcommit memory, and never check if enough memory is available. This increases the risk of out-of-memory situations, but also improves memory-intensive workloads.<br /> <br /> run echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> add 'echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled' to /etc/rc.local<br /> <br /> From here: Latency induced by transparent huge pages<br /> <br /> Unfortunately when a Linux kernel has transparent huge pages enabled, Redis incurs to a big latency penalty after the fork call is used in order to persist on disk. Huge pages are the cause of the following issue:<br /> <br /> Fork is called, two processes with shared huge pages are created.<br /> In a busy instance, a few event loops runs will cause commands to target a few thousand of pages, causing the copy on write of almost the whole process memory.<br /> This will result in big latency and big memory usage.<br /> <br /> Make sure to disable transparent huge pages using the following command:<br /> <br /> echo never &gt; /sys/kernel/mm/transparent_hugepage/enabled<br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10118 2017-05-30T18:43:56Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> screen -r sea16 on gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> cd SEAsever<br /> sh scripts/stop-sea-server.sh<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> make clean<br /> delete sea related libraries (fitcore, fpcore, libcore, seacore, seashell, seaserver) from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> <br /> <br /> # run the test:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10111 2017-05-18T21:18:38Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> ssh s_enkhee$n-1-110<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> git submodule update --init --recursive<br /> make clean<br /> delete sea related libraries from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> <br /> <br /> # run the test to check:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10071 2017-04-24T23:39:28Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submits jobs to build ligands for a given smiles file:<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ). See below the explanation of arguments and change them accordingly!:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> <br /> :b. run the script with a smiles file:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which breaks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_2017-04&diff=10070 2017-04-24T23:14:35Z

<p>Enkhjargal: Enkhjargal moved page Ligand preparation - 2017-04 to Ligand preparation - 20170424</p> <hr /> <div>#REDIRECT [[Ligand preparation - 20170424]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10069 2017-04-24T23:14:35Z

<p>Enkhjargal: Enkhjargal moved page Ligand preparation - 2017-04 to Ligand preparation - 20170424</p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submits jobs to build ligands for a given smiles file:<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> <br /> :b. run the script with a smiles file:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which breaks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10068 2017-04-24T23:13:16Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submits jobs to build ligands for a given smiles file:<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> <br /> :b. run the script with a smiles file:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which breaks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10067 2017-04-24T23:00:08Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submit jobs with :<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> :b. run the script with a smiles file:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which breaks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10066 2017-04-24T22:59:31Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submit jobs with :<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> :b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which breaks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10065 2017-04-24T22:58:35Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submit jobs with :<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> :b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which braeks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_prep_Irwin_Nov_2016&diff=10064 2017-04-24T22:57:27Z

<p>Enkhjargal: Enkhjargal moved page Ligand prep Irwin Nov 2016 to Ligand preparation - 2017-04</p> <hr /> <div>#REDIRECT [[Ligand preparation - 2017-04]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10063 2017-04-24T22:57:26Z

<p>Enkhjargal: Enkhjargal moved page Ligand prep Irwin Nov 2016 to Ligand preparation - 2017-04</p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submit jobs with :<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> :b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; Submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; But run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which braeks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10062 2017-04-24T22:56:41Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the dock environment and run the stand-alone script to build ligands for a given smiles file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.smi<br /> <br /> 2. Source the dock environment and submit jobs with :<br /> <br /> :a. create a file with following commands (in this example, the file's name is cmd ):<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 --map-instance-script \<br /> &quot;/nfs/soft/tools/utils/qsub-slice/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> :b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> What does the qsub submit script exactly do?<br /> -l 100 --&gt; Submit slices of 100 lines per task to run the script build_database_ligand.sh. <br /> -tc 10 --&gt; But run only 10 tasks at any given time. <br /> -s $BUILD_ENVIRONMENT --&gt; source the dock environment before each task is run.<br /> <br /> <br /> 3. the following is a wrapper script (wrapper_queue_build_smiles_ligand_corina.csh) which braeks up the smiles file and submits it to the queue. Users may find this script gives more control over the &quot;qsub-mr-meta&quot; way although it is less polished. <br /> #! /bin/csh<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> #set number_per_db2 = 1000<br /> #set number_per_db2 = 5<br /> set number_per_db2 = 10<br /> set ph = 7.4<br /> <br /> set file = $1<br /> set fileprefix = ${file:r}<br /> <br /> <br /> set pwd = `pwd`<br /> set pathdir = $pwd<br /> set workdir = $pwd/${fileprefix}<br /> <br /> echo ${workdir}<br /> echo ${file} ${fileprefix}<br /> <br /> if (-e ${workdir}) then<br /> echo &quot;${workdir} exists&quot;<br /> exit<br /> endif<br /> <br /> #exit<br /> #rm -rf ${workdir}<br /> mkdir ${workdir}<br /> cd ${workdir}<br /> <br /> ln -s ../${file} .<br /> <br /> echo &quot;split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split&quot;<br /> <br /> split --lines=$number_per_db2 --suffix-length=6 ${file} ${fileprefix}_split<br /> <br /> <br /> foreach splitfile ( ` ls ${fileprefix}_split* | grep -v db2.gz ` ) <br /> echo ${splitfile}<br /> <br /> # make sure that the link is pointing to something. <br /> #set lsoutput = `ls -l sgejob_*/${splitfile}.db2.gz`<br /> #echo &quot;WHAT:: $lsoutput&quot;<br /> #if (&quot;$lsoutput&quot; == &quot;&quot;) then<br /> # rm ${splitfile}.db2.gz<br /> #endif<br /> <br /> if (-e ${splitfile}.db2.gz) then<br /> echo &quot;${splitfile} has been submitted for generations.&quot; <br /> continue<br /> endif <br /> <br /> cat &lt;&lt; EOF &gt;! script_qsub_${splitfile}.csh<br /> #\$ -S /bin/csh<br /> #\$ -cwd<br /> #\$ -q all.q<br /> #\$ -o stdout_${splitfile}<br /> #\$ -e stderr_${splitfile}<br /> <br /> # source enviorment. <br /> source ~tbalius/.cshrc_dbgen_corina<br /> <br /> hostname<br /> date<br /> <br /> set SCRATCH_DIR = /scratch<br /> if ! (-d \$SCRATCH_DIR ) then<br /> SCRATCH_DIR=/tmp<br /> endif<br /> set username = `whoami`<br /> <br /> set TASK_DIR = &quot;\$SCRATCH_DIR/\${username}/\$JOB_ID&quot;<br /> echo \$TASK_DIR<br /> <br /> mkdir -p \${TASK_DIR}<br /> cd \${TASK_DIR}<br /> pwd<br /> <br /> cp ${workdir}/${splitfile} .<br /> <br /> # note that the pining script's inputs should not be in quotes ('' or &quot;&quot;).<br /> /nfs/home/tbalius/zzz.github/DOCK/common/on-one-core - ${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph ${splitfile}<br /> <br /> cd ${workdir}<br /> mkdir sgejob_\${JOB_ID}_${splitfile}<br /> <br /> echo copying<br /> ls -l \${TASK_DIR}/finished/*/*.db2.gz<br /> ls -l \${TASK_DIR}/finished/*/*.db.gz <br /> <br /> # copy finshed directory<br /> mv \${TASK_DIR}/finished/ sgejob_\${JOB_ID}_${splitfile}<br /> rm -r \${TASK_DIR}<br /> <br /> EOF<br /> <br /> while ( `qstat -u tbalius | wc -l ` &gt; 10 )<br /> sleep 10<br /> end<br /> <br /> <br /> qsub script_qsub_${splitfile}.csh<br /> <br /> #exit<br /> <br /> end<br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10061 2017-04-24T22:10:41Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> <br /> 2. In the flask code, change the database url to the newly created db <br /> <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> <br /> 4. Load the openfda regulatory_status data (populates 'Status' table in db):<br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> <br /> 5. Load the openfda excipients function data (populates 'Function' table in db):<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> <br /> 6. Load the dye relation data. (creates the relation for excipients to functions which are dye):<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> <br /> 7. Load the fda general additives data.(creates all other function and status relations to excipients): <br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> <br /> 8. Load the openfda drug label. (populate brand and substance tables and their relations to existing excipients):<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> <br /> 9. Pull Zincids for populated Substance data. (populate the zincid and smiles column in substance table)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data<br /> <br /> DB should be now fully populated.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10059 2017-04-24T22:10:13Z

<p>Enkhjargal: Enkhjargal moved page Deployment to Excipients</p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the brand new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> <br /> 2. In the flask code, change the database url to the newly created db <br /> <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> <br /> 4. Load the openfda regulatory_status data (populates 'Status' table in db):<br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> <br /> 5. Load the openfda excipients function data (populates 'Function' table in db):<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> <br /> 6. Load the dye relation data. (creates the relation for excipients to functions which are dye):<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> <br /> 7. Load the fda general additives data.(creates all other function and status relations to excipients): <br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> <br /> 8. Load the openfda drug label. (populate brand and substance tables and their relations to existing excipients):<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> <br /> 9. Pull Zincids for populated Substance data. (populate the zincid and smiles column in substance table)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data<br /> <br /> DB should be now fully populated.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10058 2017-04-24T22:04:04Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the brand new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> <br /> 2. In the flask code, change the database url to the newly created db <br /> <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> <br /> 4. Load the openfda regulatory_status data (populates 'Status' table in db):<br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> <br /> 5. Load the openfda excipients function data (populates 'Function' table in db):<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> <br /> 6. Load the dye relation data. (creates the relation for excipients to functions which are dye):<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> <br /> 7. Load the fda general additives data.(creates all other function and status relations to excipients): <br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> <br /> 8. Load the openfda drug label. (populate brand and substance tables and their relations to existing excipients):<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> <br /> 9. Pull Zincids for populated Substance data. (populate the zincid and smiles column in substance table)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data<br /> <br /> DB should be now fully populated.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10057 2017-04-24T22:03:22Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the brand new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> <br /> 2. In the flask code, change the database url to the newly created db <br /> <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> <br /> 4. Load the openfda regulatory_status data (populates 'Status' table in db):<br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> <br /> 5. Load the openfda excipients function data (populates 'Function' table in db):<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> <br /> 6. Load the dye relation data. (creates the relation for excipients to functions which are dye):<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> <br /> 7. Load the fda general additives data.(creates all other function and status relations to excipients): <br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> <br /> 8. Load the openfda drug label. (populate brand and substance tables and their relations to existing excipients):<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> <br /> 9. Pull Zincids for populated Substance data. (populate the zincid and smiles column in substance table)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10056 2017-04-24T21:53:48Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> ----<br /> <br /> [[HowTos for the brand new version of Excipients]]<br /> <br /> 1. Create a copy of existing production database<br /> 2. In the flask code, change the database url to the newly created db <br /> 3. Run db update + db migrate to create new tables and new columns for existing tables. (automatically done by flask db-migrate, mapping the model objects to postgres db) <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db upgrade<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py manage-db migrate<br /> 4. Load the openfda regulatory_status data (creates 'Status' table in db)<br /> -&gt; create a file named regulatory_status.csv<br /> touch regulatory_status.csv<br /> -&gt; copy over the regulatory_status definitions from <br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_status_definition /home/enkhjargal/PycharmProjects/Excipients/data/regulatory_status.csv<br /> 5. Load the openfda excipients function data (creates 'Function' table in db)<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_function_definition /home/enkhjargal/PycharmProjects/Excipients/data/function_definition.csv<br /> 6. Load the dye relation data:<br /> -&gt; https://www.fda.gov/ForIndustry/ColorAdditives/ColorAdditiveInventories/ucm106626.htm)<br /> -&gt; python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_color_additives_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_color_additives_all<br /> 7. Load the fda general additives data<br /> -&gt; create a file named FDA_additives<br /> touch FDA_additives<br /> -&gt; copy over the additive list from this page (https://www.fda.gov/Food/IngredientsPackagingLabeling/FoodAdditivesIngredients/ucm091048.htm)<br /> -&gt; parse the file to load it to db table<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py load_function_and_status_relationship /home/enkhjargal/PycharmProjects/Excipients/data/FDA_additives<br /> 8. Load the openfda drug label:<br /> -&gt; https://open.fda.gov/downloads/ (6 json files to download)<br /> -&gt; parse each of them:<br /> python /home/enkhjargal/PycharmProjects/Excipients/manage-excipients.py parse_fda_label_data /home/enkhjargal/PycharmProjects/Excipients/data/drug-label-0005-of-0006.json<br /> 9.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10055 2017-04-24T20:35:19Z

<p>Enkhjargal: </p> <hr /> <div>[[Excipient Server Restart:]]<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)<br /> <br /> <br /> ----<br /> <br /> [[HowTos for the brand new version of Excipients]]<br /> <br /> 1. How to load openfda data to the database:<br /> <br /> 1.</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10054 2017-04-24T20:32:39Z

<p>Enkhjargal: </p> <hr /> <div>Excipient Server Restart:<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> 2. Create the distribution file<br /> python setup.py sdist<br /> 3. ssh gimel<br /> 4. su - www<br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=10053 2017-04-24T20:29:49Z

<p>Enkhjargal: </p> <hr /> <div>Excipient Server Restart:<br /> <br /> 1. ssh gimel<br /> 2. become www<br /> 3. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> 4. gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000<br /> <br /> [[Excipient Installment:]]<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> <br /> 2. Create the distribution file<br /> python setup.py sdist<br /> <br /> 3. ssh gimel<br /> <br /> 4. su - www<br /> <br /> 5. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> <br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> <br /> 8.<br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=10048 2017-04-24T16:35:10Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> <br /> # restart conda_sea16 server<br /> <br /> ssh s_enkhee$n-1-110<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16 <br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH<br /> export PATH=/nfs/home/momeara/opt/bin:$PATH<br /> source activate sea16<br /> git pull<br /> make clean<br /> delete sea related libraries from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> <br /> # run the test to check:<br /> -&gt; export SEA_APP_ROOT=$CONDA_PREFIX/var/seaserver<br /> -&gt; export SEA_RUN_FOLDER=$SEA_APP_ROOT/run<br /> -&gt; export SEA_DATA_FOLDER=$SEA_APP_ROOT/data<br /> -&gt; python -m unittest test.test_illustrate<br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10004 2017-04-10T23:01:06Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the environment and run the stand-alone script:<br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> sh $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.sh<br /> <br /> <br /> 2. Source the environment and submit jobs:<br /> <br /> a. put these commands in cmd file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> <br /> 3. <br /> <br /> <br /> <br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10003 2017-04-10T22:54:10Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Source the environment and run the stand-alone script:<br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> $DOCKBASE/ligand/generate/build_database_ligand.sh mysmiles.sh<br /> <br /> <br /> 2. Source the environment and submit jobs:<br /> <br /> a. put these commands in cmd file:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> b. run the script:<br /> <br /> csh cmd mysmiles.smi<br /> <br /> <br /> 3. <br /> <br /> <br /> <br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10002 2017-04-10T22:41:00Z

<p>Enkhjargal: </p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Run the stand-alone script:<br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> <br /> 2. Submit jobs:<br /> <br /> setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk <br /> setenv BUILD_ENVIRONMENT /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> <br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 100 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> <br /> <br /> <br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Ligand_preparation_-_20170424&diff=10001 2017-04-10T21:47:06Z

<p>Enkhjargal: cd ..</p> <hr /> <div>== 3 ways to build ligands: ==<br /> <br /> 1. Run the stand-alone script:<br /> <br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.sh<br /> $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> 2. Submit jobs:<br /> <br /> <br /> <br /> <br /> == set up environment: jchem mitools, DOCK3.7, ZINC, corina == <br /> <br /> source /nfs/soft/jchem/current/env.csh<br /> source /nfs/soft/mitools/env.csh<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> deactivate<br /> source /nfs/soft/www/apps/zinc15/envs/edge/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh <br /> setenv ZINC_CONFIG_ENV admin<br /> setenv ZINC_CONFIG_SETUP_SKIP blueprints <br /> <br /> <br /> == step 2. csh cmd mysmiles.ism == <br /> <br /> # put any change in the BUILD_ENVIRONMENT<br /> source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh<br /> source /nfs/soft/corina/current/env.csh<br /> setenv EMBED_PROTOMERS_3D_EXE $DOCKBASE/ligand/3D/embed3d_corina.sh<br /> setenv BUILD_ENVIRONMENT /nfs/home/xyz/bin/dockenvNS.sh<br /> #<br /> /nfs/soft/tools/utils/qsub-slice/qsub-mr-meta -tc 10 -L 100000 --map-instance-script \<br /> &quot;/nfs/scratch/A/xyz/protomer/qsub-mr-map.sh&quot; -s $BUILD_ENVIRONMENT \<br /> -l 300 $1 $DOCKBASE/ligand/generate/build_database_ligand.sh<br /> <br /> If you have any trouble using this (which the procedure used to build ZINC between July 14, 2016 and Nov 14, 2016 (ongoing), <br /> please contact me. I would be happy to help debug, as it will help us to make this procedure more robust.<br /> <br /> <br /> <br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Updating_Rdkit&diff=9999 2017-04-10T18:34:14Z

<p>Enkhjargal: </p> <hr /> <div>Updating RDKIT:<br /> <br /> 1. Download RDKit Tarbal<br /> <br /> <br /> 2. Create a python virtual env:<br /> <br /> /nfs/soft/python/install/scripts/create-virtualenv.sh -s /nfs/soft/python/versions/python-2.7.11 /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> 3. Install the new version of RDKIT in the python env:<br /> <br /> /nfs/soft/python/install/scripts/install-rdkit.sh /nfs/soft/python/install/extra/RDKit_2016_03_1.tgz /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> <br /> If you need to update the boost:<br /> <br /> 1. Download the new version (http://www.boost.org/)<br /> <br /> 2. Extract the tar file and put it in /usr/local/src<br /> <br /> 3. prepare Boost for building the libraries:<br /> ./bootstrap.sh ---with-python=/nfs/soft/python/versions/pythoon-2.7.11/bin/python<br /> <br /> 4. build and install Boost:<br /> ./b2 --prefix=path_to_python_virtual_env/local/lib<br /> <br /> <br /> <br /> <br /> = Notes = See [[RDKit]]<br /> <br /> Each version is now installed in $PYTHON_ROOT/local/rdkit-VERSION. The current version is a symlink now named $PYTHON_ROOT/local/rdkit.<br /> <br /> It's worth noting how this works. There is a directory in $PYTHON_ROOT named $PYTHON_ROOT/local. In addition to containing the build of RDKit as described above ($PYTHONR_ROOT/local/rdkit), it also contains $PYTHON_ROOT/lib, which is simply a directory of simlinks to important shared libraries. <br /> <br /> When compiling Python, set LD_RUN_PATH to $ORIGIN/../lib:$ORIGIN/../local/lib so that it always looks in those two directories ($ORIGIN absolute path of the python binary). This way python can find important shared libraries, such as RDKit, modeller, etc. When we create a new virtualenv, we create a symlink to the original local directory in the virtualenv, so that we can always find the right shared libraries.<br /> <br /> To make sure we always have the right python package installed there is a file created in:<br /> $PYTHON_ROOT/lib/python2.7/site-pacakges/rdkit.pth<br /> In Python package resolution, .pth files allow you to specify an additional location to look for a module. This file includes one line:<br /> ../../../local/rdkit<br /> Which looks for the rdkit package in the (symlink to) the current version of RDKit. Note that the RDkit python package has the path:<br /> $PYTHON_ROOT/local/rdkit/rdkit (two rdkits)<br /> <br /> PTH files notes:<br /> There are actually two other pth files I have created: modeller.pth and hask.pth<br /> It would probably be best to simply create one pth file to handle these packages stored in the &quot;local&quot; directory:<br /> Create a directory called local/packages or something and put symlinks to rdkit, haks, modeller, etc. in there. then create a single pth file with the line:<br /> ../../../local/packages<br /> <br /> = Install procedure =<br /> * 0) Have a Python installation you wish to install for (This should be the base installation for virtualenvs)<br /> * 1) Download RDKit Tarball<br /> * 2) Extract RDKit Tarball: $RDKIT_SRC<br /> * 3) Download INCHI support: cd $RDKIT_SRC/External/INCHI-API; ./download-inchi.sh<br /> * 4) Create build directory: cd $RDKIT_SRC; mkdir build; cd build<br /> * 5) Configure with cmake (This is a long command! and note the version specific directories/files)<br /> <br /> cmake \<br /> -DPYTHON_EXECUTABLE=$PYTHON_PREFIX/bin/python \<br /> -DPYTHON_INCLUDE_PATH=$PYTHON_PREFIX/include/python2.7 \<br /> -DPYTHON_LIBRARY=$PYTHON_PREFIX/lib/libpython2.7.so \<br /> -DPYTHON_NUMPY_INCLUDE_PATH=$PYTHON_PREFIX/lib/python2.7/site-packages/numpy/core/include \<br /> -DRDK_BUILD_INCHI_SUPPORT=ON \<br /> -DINCHI_LIBRARY=../External/INCHI-API \<br /> -DINCHI_INCLUDE_DIR=../External/INCHI-API/src \<br /> -DCMAKE_INSTALL_PREFIX=$PYTHON_PREFIX/local/rdkit-2013.09 \<br /> <br /> * 6) Build RDKit: make -j4 (build in parallel) OR make (build serial)<br /> * 7) Wait for a while<br /> * 8) Install RDKit: make install<br /> * 9) Run python install: cd $RDKIT_SRC; $PYTHON_ROOT/bin/python setup.py install<br /> * 10) OK so the install doesn't actually work: do it manually: <br /> <br /> cd $RDKIT_SRC<br /> mkdir $PYTHON_PREFIX/local/rdkit-2013.09<br /> cp -rv lib $PYTHON_PREFIX/local/rdkit-2013.09/lib<br /> cp -rv rdkit $PYTHON_PREFIX/local/rdkit-2013.09/rdkit<br /> <br /> * 11) Update the symlinks: ln -svfn $PYTHON_PREFIX/local/rdkit-2013.09 $PYTHON_PREFIX/local/rdkit<br /> <br /> This process works but of course doesn't cover all of the details of potential installations. For one, this doesn't even touch on the upgrading of the PostgreSQL cartridge, which is a nasty black art. I'll explain that once I have enough goats to sacrifice.<br /> <br /> [[Category:Sysadmin]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Updating_Rdkit&diff=9998 2017-04-10T18:21:50Z

<p>Enkhjargal: </p> <hr /> <div>Download RDKit Tarbal<br /> <br /> <br /> Create a python virtual env:<br /> <br /> /nfs/soft/python/install/scripts/create-virtualenv.sh -s /nfs/soft/python/versions/python-2.7.11 /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> Install the new version of RDKIT in the python env:<br /> <br /> /nfs/soft/python/install/scripts/install-rdkit.sh /nfs/soft/python/install/extra/RDKit_2016_03_1.tgz /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> <br /> = Notes = See [[RDKit]]<br /> <br /> Each version is now installed in $PYTHON_ROOT/local/rdkit-VERSION. The current version is a symlink now named $PYTHON_ROOT/local/rdkit.<br /> <br /> It's worth noting how this works. There is a directory in $PYTHON_ROOT named $PYTHON_ROOT/local. In addition to containing the build of RDKit as described above ($PYTHONR_ROOT/local/rdkit), it also contains $PYTHON_ROOT/lib, which is simply a directory of simlinks to important shared libraries. <br /> <br /> When compiling Python, set LD_RUN_PATH to $ORIGIN/../lib:$ORIGIN/../local/lib so that it always looks in those two directories ($ORIGIN absolute path of the python binary). This way python can find important shared libraries, such as RDKit, modeller, etc. When we create a new virtualenv, we create a symlink to the original local directory in the virtualenv, so that we can always find the right shared libraries.<br /> <br /> To make sure we always have the right python package installed there is a file created in:<br /> $PYTHON_ROOT/lib/python2.7/site-pacakges/rdkit.pth<br /> In Python package resolution, .pth files allow you to specify an additional location to look for a module. This file includes one line:<br /> ../../../local/rdkit<br /> Which looks for the rdkit package in the (symlink to) the current version of RDKit. Note that the RDkit python package has the path:<br /> $PYTHON_ROOT/local/rdkit/rdkit (two rdkits)<br /> <br /> PTH files notes:<br /> There are actually two other pth files I have created: modeller.pth and hask.pth<br /> It would probably be best to simply create one pth file to handle these packages stored in the &quot;local&quot; directory:<br /> Create a directory called local/packages or something and put symlinks to rdkit, haks, modeller, etc. in there. then create a single pth file with the line:<br /> ../../../local/packages<br /> <br /> = Install procedure =<br /> * 0) Have a Python installation you wish to install for (This should be the base installation for virtualenvs)<br /> * 1) Download RDKit Tarball<br /> * 2) Extract RDKit Tarball: $RDKIT_SRC<br /> * 3) Download INCHI support: cd $RDKIT_SRC/External/INCHI-API; ./download-inchi.sh<br /> * 4) Create build directory: cd $RDKIT_SRC; mkdir build; cd build<br /> * 5) Configure with cmake (This is a long command! and note the version specific directories/files)<br /> <br /> cmake \<br /> -DPYTHON_EXECUTABLE=$PYTHON_PREFIX/bin/python \<br /> -DPYTHON_INCLUDE_PATH=$PYTHON_PREFIX/include/python2.7 \<br /> -DPYTHON_LIBRARY=$PYTHON_PREFIX/lib/libpython2.7.so \<br /> -DPYTHON_NUMPY_INCLUDE_PATH=$PYTHON_PREFIX/lib/python2.7/site-packages/numpy/core/include \<br /> -DRDK_BUILD_INCHI_SUPPORT=ON \<br /> -DINCHI_LIBRARY=../External/INCHI-API \<br /> -DINCHI_INCLUDE_DIR=../External/INCHI-API/src \<br /> -DCMAKE_INSTALL_PREFIX=$PYTHON_PREFIX/local/rdkit-2013.09 \<br /> <br /> * 6) Build RDKit: make -j4 (build in parallel) OR make (build serial)<br /> * 7) Wait for a while<br /> * 8) Install RDKit: make install<br /> * 9) Run python install: cd $RDKIT_SRC; $PYTHON_ROOT/bin/python setup.py install<br /> * 10) OK so the install doesn't actually work: do it manually: <br /> <br /> cd $RDKIT_SRC<br /> mkdir $PYTHON_PREFIX/local/rdkit-2013.09<br /> cp -rv lib $PYTHON_PREFIX/local/rdkit-2013.09/lib<br /> cp -rv rdkit $PYTHON_PREFIX/local/rdkit-2013.09/rdkit<br /> <br /> * 11) Update the symlinks: ln -svfn $PYTHON_PREFIX/local/rdkit-2013.09 $PYTHON_PREFIX/local/rdkit<br /> <br /> This process works but of course doesn't cover all of the details of potential installations. For one, this doesn't even touch on the upgrading of the PostgreSQL cartridge, which is a nasty black art. I'll explain that once I have enough goats to sacrifice.<br /> <br /> [[Category:Sysadmin]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Updating_Rdkit&diff=9997 2017-04-10T18:08:27Z

<p>Enkhjargal: </p> <hr /> <div>Create a python virtual env:<br /> <br /> /nfs/soft/python/install/scripts/create-virtualenv.sh -s /nfs/soft/python/versions/python-2.7.11 /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> Install the new version of RDKIT in the python env:<br /> <br /> /nfs/soft/python/install/scripts/install-rdkit.sh /nfs/soft/python/install/extra/RDKit_2016_03_1.tgz /nfs/soft/python/envs/rdkit/python-2.7.11-rdkit-2016_03_01<br /> <br /> <br /> = Notes = See [[RDKit]]<br /> <br /> Each version is now installed in $PYTHON_ROOT/local/rdkit-VERSION. The current version is a symlink now named $PYTHON_ROOT/local/rdkit.<br /> <br /> It's worth noting how this works. There is a directory in $PYTHON_ROOT named $PYTHON_ROOT/local. In addition to containing the build of RDKit as described above ($PYTHONR_ROOT/local/rdkit), it also contains $PYTHON_ROOT/lib, which is simply a directory of simlinks to important shared libraries. <br /> <br /> When compiling Python, set LD_RUN_PATH to $ORIGIN/../lib:$ORIGIN/../local/lib so that it always looks in those two directories ($ORIGIN absolute path of the python binary). This way python can find important shared libraries, such as RDKit, modeller, etc. When we create a new virtualenv, we create a symlink to the original local directory in the virtualenv, so that we can always find the right shared libraries.<br /> <br /> To make sure we always have the right python package installed there is a file created in:<br /> $PYTHON_ROOT/lib/python2.7/site-pacakges/rdkit.pth<br /> In Python package resolution, .pth files allow you to specify an additional location to look for a module. This file includes one line:<br /> ../../../local/rdkit<br /> Which looks for the rdkit package in the (symlink to) the current version of RDKit. Note that the RDkit python package has the path:<br /> $PYTHON_ROOT/local/rdkit/rdkit (two rdkits)<br /> <br /> PTH files notes:<br /> There are actually two other pth files I have created: modeller.pth and hask.pth<br /> It would probably be best to simply create one pth file to handle these packages stored in the &quot;local&quot; directory:<br /> Create a directory called local/packages or something and put symlinks to rdkit, haks, modeller, etc. in there. then create a single pth file with the line:<br /> ../../../local/packages<br /> <br /> = Install procedure =<br /> * 0) Have a Python installation you wish to install for (This should be the base installation for virtualenvs)<br /> * 1) Download RDKit Tarball<br /> * 2) Extract RDKit Tarball: $RDKIT_SRC<br /> * 3) Download INCHI support: cd $RDKIT_SRC/External/INCHI-API; ./download-inchi.sh<br /> * 4) Create build directory: cd $RDKIT_SRC; mkdir build; cd build<br /> * 5) Configure with cmake (This is a long command! and note the version specific directories/files)<br /> <br /> cmake \<br /> -DPYTHON_EXECUTABLE=$PYTHON_PREFIX/bin/python \<br /> -DPYTHON_INCLUDE_PATH=$PYTHON_PREFIX/include/python2.7 \<br /> -DPYTHON_LIBRARY=$PYTHON_PREFIX/lib/libpython2.7.so \<br /> -DPYTHON_NUMPY_INCLUDE_PATH=$PYTHON_PREFIX/lib/python2.7/site-packages/numpy/core/include \<br /> -DRDK_BUILD_INCHI_SUPPORT=ON \<br /> -DINCHI_LIBRARY=../External/INCHI-API \<br /> -DINCHI_INCLUDE_DIR=../External/INCHI-API/src \<br /> -DCMAKE_INSTALL_PREFIX=$PYTHON_PREFIX/local/rdkit-2013.09 \<br /> <br /> * 6) Build RDKit: make -j4 (build in parallel) OR make (build serial)<br /> * 7) Wait for a while<br /> * 8) Install RDKit: make install<br /> * 9) Run python install: cd $RDKIT_SRC; $PYTHON_ROOT/bin/python setup.py install<br /> * 10) OK so the install doesn't actually work: do it manually: <br /> <br /> cd $RDKIT_SRC<br /> mkdir $PYTHON_PREFIX/local/rdkit-2013.09<br /> cp -rv lib $PYTHON_PREFIX/local/rdkit-2013.09/lib<br /> cp -rv rdkit $PYTHON_PREFIX/local/rdkit-2013.09/rdkit<br /> <br /> * 11) Update the symlinks: ln -svfn $PYTHON_PREFIX/local/rdkit-2013.09 $PYTHON_PREFIX/local/rdkit<br /> <br /> This process works but of course doesn't cover all of the details of potential installations. For one, this doesn't even touch on the upgrading of the PostgreSQL cartridge, which is a nasty black art. I'll explain that once I have enough goats to sacrifice.<br /> <br /> [[Category:Sysadmin]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Adverse&diff=9976 2017-03-23T17:05:16Z

<p>Enkhjargal: </p> <hr /> <div>main web page -&gt; <br /> index.html:<br /> <br /> <br /> splashes -&gt;<br /> Images/Icons/Android<br /> Images/Icons/iOS/Splash/<br /> <br /> themes -&gt;<br /> css files<br /> <br /> organ objects:<br /> int-organs.js<br /> ext-organs.js<br /> int-spots.js<br /> ext-spots.js<br /> <br /> body diagram interactions, functions using organ objects:<br /> interact-script.js<br /> <br /> objects of all adverse event terms:<br /> meddra.js<br /> <br /> search in openfda and draw graph with loader.js:<br /> fdajson.js <br /> <br /> user profile<br /> profile.js <br /> <br /> phonegap build config (important is to change the version number)<br /> config.xml<br /> <br /> how to build the app:<br /> https://build.phonegap.com/<br /> -&gt; create an account<br /> -&gt; zip the code folder and upload on this website<br /> -&gt; .apk is for android<br /> -&gt; .ipa is for ios<br /> create a special key for each OS<br /> <br /> how to upload the app:<br /> developers.google.com/apps<br /> itunes connect</div>

Enkhjargal http://wiki.docking.org/index.php?title=Adverse&diff=9975 2017-03-23T16:22:59Z

<p>Enkhjargal: </p> <hr /> <div>index.html:<br /> main web page. <br /> <br /> splashes -&gt;<br /> Images/Icons/Android<br /> Images/Icons/iOS/Splash/<br /> <br /> themes -&gt;<br /> css files<br /> <br /> organ objects:<br /> int-organs.js<br /> ext-organs.js<br /> int-spots.js<br /> ext-spots.js<br /> <br /> body diagram interactions, functions using organ objects:<br /> interact-script.js<br /> <br /> objects of all adverse event terms:<br /> meddra.js<br /> <br /> search in openfda and draw graph with loader.js:<br /> fdajson.js <br /> <br /> user profile<br /> profile.js <br /> <br /> phonegap build config (important is to change the version number)<br /> config.xml<br /> <br /> how to build the app:<br /> https://build.phonegap.com/<br /> -&gt; create an account<br /> -&gt; zip the code folder and upload on this website<br /> -&gt; .apk is for android<br /> -&gt; .ipa is for ios<br /> create a special key for each OS<br /> <br /> how to upload the app:<br /> developers.google.com/apps<br /> itunes connect</div>

Enkhjargal http://wiki.docking.org/index.php?title=Adverse&diff=9973 2017-03-22T22:16:51Z

<p>Enkhjargal: Created page with &quot;index.html: main web page. splashes -&gt; Images/Icons/Android Images/Icons/iOS/Splash/ themes -&gt; css files organ objects: int-organs.js ext-organs.js int-spots.js ex...&quot;</p> <hr /> <div>index.html:<br /> main web page. <br /> <br /> splashes -&gt;<br /> Images/Icons/Android<br /> Images/Icons/iOS/Splash/<br /> <br /> themes -&gt;<br /> css files<br /> <br /> organ objects:<br /> int-organs.js<br /> ext-organs.js<br /> int-spots.js<br /> ext-spots.js<br /> <br /> body diagram interactions, functions using organ objects:<br /> interact-script.js<br /> <br /> objects of all adverse event terms:<br /> meddra.js<br /> <br /> search in openfda and draw graph with loader.js:<br /> fdajson.js <br /> <br /> user profile<br /> profile.js <br /> <br /> phonegap build config (important is to change the version number)<br /> config.xml</div>

Enkhjargal http://wiki.docking.org/index.php?title=DOCK3.7_INDOCK_Minimization_Parameter&diff=9969 2017-03-17T20:48:59Z

<p>Enkhjargal: </p> <hr /> <div>The best scored ligand conformation can be minimized using the following parameters.<br /> <br /> Add following section to existing INDOCK after SCORING section.<br /> <br /> ##################################################### <br /> # MINIMIZATION<br /> minimize yes<br /> sim_itmax 500<br /> sim_trnstep 0.2<br /> sim_rotstep 5.0<br /> sim_need_to_restart 1.0<br /> sim_cnvrge 0.1<br /> min_cut 1.0e15<br /> iseed 777<br /> <br /> <br /> The default for minimize is no. Yes turns on the minimization.<br /> minimize yes<br /> <br /> How many iterations of minimization to do. More means longer run times, but potentially better poses. <br /> <br /> sim_itmax 500<br /> <br /> This is the initial distance in angstroms the molecule is translated (In Angstrom). (This is used to initiliaze the simplex)<br /> <br /> sim_trnstep 0.2<br /> <br /> How many degrees of initial rotation are done (In Degree). (This is used to initiliaze the simplex)<br /> <br /> sim_rotstep 5.0<br /> <br /> If the energy changes by this much, restart the minimizer from this newest position. Between 2 subseq runs: if energy changes by more than 1kcal restart minimizer from latest position for another round.<br /> sim_need_to_restart 1.0<br /> <br /> How much the total energy can changed to be considered converged. within one run: considered converged when energies of lowest and highest energy state differ by only 0.1kcal/mol. Setting this higher will stop faster, setting it lower will cause it to do more iterations before converging (or potentially hitting the iteration max above).<br /> sim_cnvrge 0.1<br /> <br /> Don't minimize molecules that score above this large number. <br /> min_cut 1.0e15<br /> <br /> To initiliaze the simplex, this iseed number used to generate a random number. e.g. used for trn and rotstep calc <br /> iseed 777</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=9968 2017-03-16T21:14:34Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if it not works: export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> git pull<br /> make clean<br /> delete sea related libraries from your site-packages folder under the conda env (rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver)<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh s_enkhee$n-1-110<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if not works, export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=9967 2017-03-15T16:55:26Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if not works, export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver<br /> <br /> git pull<br /> make clean<br /> delete sea related libraries from your site-packages folder under the conda env<br /> make all<br /> kill all sea-server related processes e.g. from htop<br /> make SEAserver-start<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh s_enkhee$n-1-110<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if not works, export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=Sea16_restart&diff=9963 2017-03-09T19:51:59Z

<p>Enkhjargal: </p> <hr /> <div>&lt;pre&gt;<br /> <br /> # update conda_sea16<br /> <br /> ssh s_enkhee@gimel<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if not works, export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh s_enkhee$n-1-110<br /> cd /nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/src/seaware-academic/<br /> source activate sea16 (if not works, export PATH=/nfs/soft/www/apps/sea/conda_sea16/anaconda2/envs/sea16/bin:$PATH)<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> <br /> <br /> <br /> # update sea16<br /> ssh xyz@gimel<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> git pull<br /> rm -rf /nfs/soft/www/apps/sea/sea16/lib/python2.7/site-packages/seaserver<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic/SEAserver<br /> python setup.py install<br /> <br /> # restart server<br /> <br /> ssh www@n-1-110<br /> cd /nfs/soft/www/apps/sea/sea16/src/seaware-academic<br /> source ../../env.csh<br /> cd SEAsever<br /> sh scripts/run-sea-server.sh<br /> <br /> # restart server<br /> <br /> ssh &lt;superuser&gt;@n-1-110<br /> sudo -i<br /> screen -r<br /> screen -dR Sea (switch to sea screen)<br /> sh scripts/run-sea-server.sh<br /> <br /> # how to save the old queue data<br /> <br /> cd /nfs/soft/www/apps/sea/sea16/var/seaserver/queue <br /> mv jobs jobs.save<br /> mv tasks.sqlite tasks.sqllite.save <br /> restart sea server on n-1-110. <br /> <br /> (basically, it had too much history and that was what was slowing it down)<br /> (do on the first day of the month and rename the old one to a month version)<br /> <br /> <br /> &lt;/pre&gt;<br /> <br /> [[Category:Curator]]</div>

Enkhjargal http://wiki.docking.org/index.php?title=AWS_General_Notes&diff=9936 2017-02-27T22:31:21Z

<p>Enkhjargal: Created page with &quot;&#039;&#039;&#039;Modifying an EBS Volume from the Console:&#039;&#039;&#039; The following procedure shows how to apply available volume modifications from the Amazon EC2 console. 1. Open the Amazon...&quot;</p> <hr /> <div>'''Modifying an EBS Volume from the Console:'''<br /> <br /> The following procedure shows how to apply available volume modifications from the Amazon EC2 console.<br /> <br /> 1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.<br /> <br /> 2. Choose Volumes, select the volume to modify and then choose Actions, Modify Volume.<br /> <br /> 3. The Modify Volume window displays the volume ID and the volume's current configuration, including type, size, and IOPS. You can change any or all of these settings in a single action. Set new configuration values as follows:<br /> <br /> -&gt; To modify the type, choose a value for Volume Type.<br /> <br /> -&gt; To modify the size, enter an allowed integer value for Size.<br /> <br /> -&gt; If you chose Provisioned IOPS (IO1) as your volume type, enter an allowed integer value for IOPS.<br /> <br /> 4. After you have specified all of the modifications to apply, choose Modify, Yes. <br /> <br /> <br /> Modifying volume size has no practical effect until you also extend the volume's file system to make use of the new storage capacity.<br /> <br /> '''To extend a Linux file system:'''<br /> <br /> 1. Log in to your Linux instance using an SSH client.<br /> <br /> 2. Use the df -h command to see the existing file system disk space usage.<br /> df -h <br /> <br /> 3. Use a file system-specific command to resize each file system to the new volume capacity. <br /> sudo resize2fs /dev/xvda1</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=9932 2017-02-21T21:26:58Z

<p>Enkhjargal: </p> <hr /> <div>Excipient Deployment:<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> <br /> 2. Create the distribution file<br /> python setup.py sdist<br /> <br /> 3. <br /> ssh &lt;super_user&gt;@gimel<br /> <br /> 4. <br /> sudo -i<br /> <br /> 5. <br /> su - www<br /> <br /> 4. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> <br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> <br /> 8. find the excipients screen -&gt; <br /> screen -ls<br /> <br /> 9. switch to excipients screen -&gt; <br /> screen -dR excip<br /> <br /> 10. CTRL-C to kill<br /> <br /> 11. &lt;UP&gt; &lt;ENTER&gt; to rerun<br /> (this will start the gunicorn server: <br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5 --timeout 1000)</div>

Enkhjargal http://wiki.docking.org/index.php?title=Excipient_server_restart&diff=9923 2017-02-09T19:56:35Z

<p>Enkhjargal: </p> <hr /> <div>Excipient Deployment:<br /> <br /> 1. Go to the dev version of Excipients and set the version number set in the __init__.py file<br /> <br /> 2. Create the distribution file<br /> python setup.py sdist<br /> <br /> 3. <br /> ssh &lt;super_user&gt;@gimel<br /> <br /> 4. <br /> sudo -i<br /> <br /> 5. <br /> su - www<br /> <br /> 4. activate the production server -&gt;<br /> cd /nfs/soft/www/apps/excipients/envs/production/<br /> source bin/activate<br /> <br /> 6. copy over the created dist folder in /nfs/soft/www/apps/excipients/envs/production/ <br /> <br /> 7. Run -&gt; <br /> pip install dist/CERSI-Excipients-X.Y.Z.tar.gz<br /> <br /> 8. find the excipients screen -&gt; <br /> screen -ls<br /> <br /> 9. switch to excipients screen -&gt; <br /> screen -dR excip<br /> <br /> 10. CTRL-C to kill<br /> <br /> 11. &lt;UP&gt; &lt;ENTER&gt; to rerun<br /> (this will start the gunicorn server: <br /> gunicorn --access-logfile excipients.ucsf.bkslab.org.acc --max-requests 1000 --bind 10.20.0.31:8093 excipients:app --workers 5)</div>

Enkhjargal