SmallWorld Documentation for Future Developer: Difference between revisions

From DISI
Jump to navigation Jump to search
 
(38 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Introduction ==
= Introduction =
[https://www.nextmovesoftware.com/downloads/smallworld/documentation/SmallWorld.pdf Here is the link to Smallworld's manual]
[https://www.nextmovesoftware.com/downloads/smallworld/documentation/SmallWorld.pdf Here is the link to Smallworld's manual]
* Username: ucsf@nextmovesoftware.com
* Username: ucsf@nextmovesoftware.com
Line 10: Line 10:
Execute command '''start_smallworld''' to see options on which smallworld instance to start/restart.
Execute command '''start_smallworld''' to see options on which smallworld instance to start/restart.


== How to Download Smallworld ==
= How to Download Smallworld =
# Ssh to abacus and become root. Prepare directory
# Ssh to abacus and become root. Prepare directory
#: <source> mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version> </source>
#: <source> mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version> </source>
Line 22: Line 22:
#* <source> tar -xvf <file_name> </source>
#* <source> tar -xvf <file_name> </source>


== How to Launch Smallworld For The First Time ==
= How to Launch Smallworld For The First Time =
=== Prepare Files and Directories ===
== Prepare Files and Directories ==
# Ssh to abacus and become root
# Ssh to abacus and become root
# Open a port in the firewall
# Open a port in the firewall
Line 72: Line 72:
#* '''SearchTimeOut''' should be the only option you'll change in the future. It caps search time to lessen strain on machine.
#* '''SearchTimeOut''' should be the only option you'll change in the future. It caps search time to lessen strain on machine.
#Place any *.anon.maps you've already compiled for testing into the maps sub-directory.
#Place any *.anon.maps you've already compiled for testing into the maps sub-directory.
=== Start Smallworld Instance ===
== Start Smallworld Instance ==
# Go to smallworld extract directory
# Go to smallworld extract directory
#: <source>cd /export/db3/smallworld_extract</source>
#: <source>cd /export/db3/smallworld_extract</source>
Line 82: Line 82:
#: <source>java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number></source>
#: <source>java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number></source>


== How to Build Smallworld Maps/Databases ==
= How to Build Smallworld Maps/Databases =
Consolidate smiles into '''/export/db3/temp_mapping'''.
Consolidate smiles into '''/export/db3/temp_mapping'''.


Line 103: Line 103:
end
end
</source>
</source>
=== Command Details ===
== Command Details ==
'''setenv''' sets the environment where the anon master map is
'''setenv''' sets the environment where the anon master map is
'''java -jar /opt/nextmove/smallworld-5.1/sw.jar''' shortened to '''sw''' via alias.
 
'''java -jar /opt/nextmove/smallworld-5.1/sw.jar''' shortened to '''sw''' via alias
 
'''map''' will map the smiles based on the current anon master map.
'''map''' will map the smiles based on the current anon master map.
*'''-T''' stores temp files so /tmp does not get overloaded
*'''-T''' stores temp files so /tmp does not get overloaded
Line 112: Line 114:
'''tee''' will log the progress of the process
'''tee''' will log the progress of the process


=== Splitting Databases ===
== Splitting Databases ==
[[http://wiki.docking.org/index.php/Creating_Maps_on_SmallWorld| Follow this page if John has asked you to split the database into smaller slices.]]
[[http://wiki.docking.org/index.php/Creating_Maps_on_SmallWorld| Follow this page if John has asked you to split the database into smaller slices.]]


==Running SmallWorld from the Command Line==
= Active Smallworld Instances =
We have been running SmallWorld on n-9-22 as www which uses csh as the default shell.
{| class="wikitable"
There is a file in /mnt/nfs/soft/www/home/smallworld called run_sw which contains the following commands to set up the SmallWorld Command Line Interface (CLI)
|+ Caption text
|-
! Type !! URL !! Machine:Port !! Directory
|-
| Public || http://sw.docking.org/ || 10.20.0.5:5020 || /nfs/db3/public_smallworld_4th_gen
|-
| Private || http://swp.docking.org/ || 10.20.0.5:8080 || /nfs/db3/private_smallworld_4th_gen
|-
| Super Private || http://swc.docking.org/ || 10.20.0.5:5099 || /nfs/db3/super_private_smallworld_4th_gen
|-
| SWBB || http://swbb.docking.org/ || 10.20.0.5:8081 || /nfs/db3/swbb_5th_gen
|-
| SWCC || http://swcc.docking.org/ || 10.20.0.5:8082 || /nfs/db3/swcc_5th_gen
|-
|}
 
= Customize Smallworld FrontEnd To Our Needs (Smallworld 5.5) =
'''These instructions only worked and compiled for me in the machine called epyc which is running Rocky 8 Linux operating system.'''
 
Summary of changes in swsearch.js:
* Conditional hyperlinks attached to smallworld hits. Leads to zinc20, cartblanche22, or nothing.
* Adding clipboard icon to copy SMILES of hits.
 
Summary of changes in search.html:
* Add 'Ask Questions' and contact info
* Add 'Database Information' hyperlink
* Pre-toggle Advanced Option
 
Summary of changes in smallworld-swagger.yaml:
* Change API route to be just "/"
 
== Install Prerequisite Packages ==
# Install Apache Maven
#* <source>dnf install maven -y</source>
# In your home directory, create a new directory to hold the files for the upcoming procedures
#* <source>mkdir /mnt/nfs/home/jjg/smallworld_build_from_source</source>
# Download these latest Smallworlds and store them in 'smallworld_build_from_source/'. [[SmallWorld_Documentation_for_Future_Developer#How_to_Download_Smallworld|Here's how to download Smallworld.]]
#* smallworld-java-5.5.tar.gz
#* smallworld-java-src-5.5.tar.gz
# Extract contents from the tar.gz files
#* <source>tar -xvf smallworld-java-5.5.tar.gz
tar -xvf smallworld-java-src-5.5.tar.gz</source>
# Install Apache Maven Smallworld Dependencies
#* <source>
cd /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java/lib/
bash mvn-install.sh
</source>
 
== Customizing swsearch.js ==
Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js


Set the SWDIR environment variable so SmallWorld scripts can access the 12 TB index
=== Conditional Hyperlinks ===
    setenv SWDIR /srv/db4
# vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js
# Search for this
#* <source>?div.append("<b>" + id + "</b>");</source>
# Replace the inside of the else statement with these lines of code
#* <source>
                if (id.substring(0,6) == "ZINC00") {
                        div.append("<b><a target='_blank' href='https://zinc20.docking.org/substances/"+id+"'>" + id + "</a></b>");
                } else if(id.substring(0,4) == "ZINC" && id.substring(4,6) != "00") {
                        div.append("<b><a target='_blank' href='https://cartblanche22.docking.org/substance/"+id+"'>" + id + "</a></b>");
                } else {
                        div.append("<b>" + id + "</b>");
                }
</source>
=== Copy SMILES of Hits ===
# Search for this
#: <source>?var $props</source>
# Add this line at the end of the appends
#: <source>
                              .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
</source>
# It should look like this
#: <source>
        var $props = $('<div>').append('<b>SWIDX:</b> ' + data.anonIdx)
                              .append('<br/><b>MW:</b> ' + data.mw)
                              .append('<br/><b>MF:</b> ' + data.mf)
                              .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
</source>
# Then add this function anywhere except inside another function's scope
#: <source>
function copySmiles(smiles) {
        navigator.clipboard.writeText(smiles);
}
</source>
 
== Customizing search.html ==
Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
 
=== Add 'Ask Questions' and 'Database Information' ===
# vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
# Search 'Scoring Methods'
#: <source>?Scoring Methods</source>
# At the end of that big block of 'div', add these:
#: <source>
<div class="section-head">
        Ask Questions
        <table class="swopt searchasyoudraw" style="width: 100%">
                <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
        </table>
</div>
<div class="section-head">
        Database Descriptions
        <table class="swopt searchasyoudraw" style="width: 100%">
                <th style="text-align: left">
                      <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                </th>
        </table>
</div>
</source>
# It should look like this
#: <source>
            <div class="section-head">
                Scoring Methods
            </div>
            <table id="swscore">
 
            </table>
        </div>
        <div class="section-head">
                Ask Questions
                <table class="swopt searchasyoudraw" style="width: 100%">
                        <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
                </table>
        </div>
        <div class="section-head">
                Database Descriptions
                <table class="swopt searchasyoudraw" style="width: 100%">
                        <th style="text-align: left">
                                <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                        </th>
                </table>
        </div>
</source>
=== Pre-toggle Advanced Option ===
# Inside the script element, add this function
#: <source>
window.onload = function() {
        $('#advPanel').css('display', $(this).prop('checked', true) && 'block'); update_visible_columns();
}
</source>
# Then search for "update_visible_columns" and scroll till you find the input checkbox element
#:<source>?update_visible_columns</source>
# In the input box element, add the checked option. Should look like this
#:<source>
<input name="optAdv" type="checkbox"
                          onchange="$('#advPanel').css('display', $(this).prop('checked') ? 'block' : 'none'); update_visible_columns();" checked>
</source>
 
=== Change API Route to "/" ===
# path smallworld-java-src/web/target/sw-ws-5.6.1/WEB-INF/static/swagger/smallworld-swagger.yaml
 
== Compile Code To Make WAR File ==
#Build WAR File
#: <source>
cd /nfs/home/jjg/smallworld_build_from_source/smallworld-java-src
mvn install -Pbootable
</source>
#If it works, then the new WAR File is located at '''/mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/target/sw-ws-5.5.war'''
 
= Restarting Smallworld Instance(s) =
=== Old Way ===
# Ssh to abacus and become root
# execute '''run_sw_on_reboot.sh''' to show restart all instances on the machine
#: <source>
bash /root/run_arthors_on_reboot.sh
</source>
# execute '''start_smallworld''' to restart specific Smallworld instance. It will show you options to choose from.
#: <source>
start_smallworld
</source>
=== New Way ===
# Ssh to abacus and become root. Then run
#: <source>
systemctl <start|stop|restart> <sw|swp|swc|swbb|swcc>
</source>


Set the path to the SmallWorld C++ Executables
    set path = ($path /opt/nextmove/bin)
Alias the keyword "sw" to access the Java CLI
    alias sw 'java -jar /opt/nextmove/sw.jar'


==Tips for Running SmallWorld on the Command Line==
[[Category:Smallworld]]
Use sw help to get information about available commands
[[Category:Developer]]
    sw help

Latest revision as of 00:57, 25 June 2024

Introduction

Here is the link to Smallworld's manual

  • Username: ucsf@nextmovesoftware.com
  • Password: <Ask jjiteam@googlegroups.com>

All smallworld databases live in abacus' db3.

Smallworld frontend files are consolidated and located at /export/db3/smallworld_extract/.extract.

Execute command start_smallworld to see options on which smallworld instance to start/restart.

How to Download Smallworld

  1. Ssh to abacus and become root. Prepare directory
     mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version>
  2. Download Software with this link
    • Username: ucsf@nextmovesoftware.com
    • Password: <Ask jjiteam@googlegroups.com>
  3. Go to releases. Look for smallworld-java-<version>.tar.gz and copy the link address.
  4. Download using wget
     wget --user ucsf@nextmovesoftware.com --password <Ask jjiteam@googlegroups.com> <link address>
  5. Decompress the file
    •  tar -xvf <file_name>

How to Launch Smallworld For The First Time

Prepare Files and Directories

  1. Ssh to abacus and become root
  2. Open a port in the firewall
    firewall-cmd --permanent --add-port=<port_number>/tcp 
    firewall-cmd --reload
  3. Go to smallworld databases directory
    cd /export/db3/
  4. Create a Smallworld directory and go to it
    mkdir <smallworld_dir> && cd <smallworld_dir>
  5. Inside that directory, create these
    • a symbolic link to the smallworld anon map
    ln -s /export/db2/smallworld_anon_21Q4 anon
    • a sub-directory called maps to store databases
    mkdir maps
    • a smallworld config file
    vim smallworld.cfg
    • Add this block of code inside the config file
    [WebApp]
    DepictionUrl=./depict/svg?w=%w&h=%h&smi=%s&qry=%q&cols=%c&cmap=%m
    ResolverUrl=./util/smi2mol?smi=%s
    SearchTimeOut=5000
    SearchAsYouDraw=true
    [Score]
    Name=Atom Alignment
    Description=Aligns and scores each hit relative to the query atom types, differences are categorised as MAJ, MIN, HYB, SUB 
    Class=com.nmsoftware.smallworld.score.SimpleAtomAlignment 
    EnabledByDefault=false
    MappingMode=EditMap
    [Score]
    Name=SMARTS Alignment
    Description=Treat input as SMARTS and align and score each hit relative to the query atom expressions in the input 
    Class=com.nmsoftware.smallworld.score.SmartsAtomAlignment 
    EnabledByDefault=false
    [Score]
    Name=ECFP4
    Description=Extended Connectivity Circular Fingerprint
    Class=com.nmsoftware.smallworld.score.EcfpTanimoto
    Radius=2
    EnabledByDefault=true
    [Score]
    Name=Daylight
    Description=Path based fingerprint
    Class=com.nmsoftware.smallworld.score.DaylightTanimoto
    EnabledByDefault=true
    • SearchTimeOut should be the only option you'll change in the future. It caps search time to lessen strain on machine.
  6. Place any *.anon.maps you've already compiled for testing into the maps sub-directory.

Start Smallworld Instance

  1. Go to smallworld extract directory
    cd /export/db3/smallworld_extract
  2. Create a screen
    screen -S <screen_name>
  3. Export the smallworld directory you've made
    export SWDIR=/export/db3/<smallworld_dir>
  4. Launch java
    java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number>

How to Build Smallworld Maps/Databases

Consolidate smiles into /export/db3/temp_mapping.

Use the script located in that directory called smallworld_map_script.

Here is the content of the script:

#!/bin/csh -f
setenv SWDIR /export/db3/public_smallworld_4th_gen/

alias sw 'time java -jar /opt/nextmove/smallworld-5.1/sw.jar'

set target = '*.smi'

foreach i ($target)
	echo '(sw map' $i '-T tmp/ -o '${i}'.anon.map -u '${i}'.anon.unmapped) |& tee '${i}'_log'
	(sw map $i -T tmp/ -o ${i}.anon.map -u ${i}.anon.unmapped) |& tee ${i}_log
	echo 'sw blm -p 0.01 '${i}'.anon.map'
	sw blm -p 0.01 ${i}.anon.map
end

Command Details

setenv sets the environment where the anon master map is

java -jar /opt/nextmove/smallworld-5.1/sw.jar shortened to sw via alias

map will map the smiles based on the current anon master map.

  • -T stores temp files so /tmp does not get overloaded
  • -o output with custom name you like
  • -u will collect the unmapped molecules to send to Nextmove or to use swextend

tee will log the progress of the process

Splitting Databases

[Follow this page if John has asked you to split the database into smaller slices.]

Active Smallworld Instances

Caption text
Type URL Machine:Port Directory
Public http://sw.docking.org/ 10.20.0.5:5020 /nfs/db3/public_smallworld_4th_gen
Private http://swp.docking.org/ 10.20.0.5:8080 /nfs/db3/private_smallworld_4th_gen
Super Private http://swc.docking.org/ 10.20.0.5:5099 /nfs/db3/super_private_smallworld_4th_gen
SWBB http://swbb.docking.org/ 10.20.0.5:8081 /nfs/db3/swbb_5th_gen
SWCC http://swcc.docking.org/ 10.20.0.5:8082 /nfs/db3/swcc_5th_gen

Customize Smallworld FrontEnd To Our Needs (Smallworld 5.5)

These instructions only worked and compiled for me in the machine called epyc which is running Rocky 8 Linux operating system.

Summary of changes in swsearch.js:

  • Conditional hyperlinks attached to smallworld hits. Leads to zinc20, cartblanche22, or nothing.
  • Adding clipboard icon to copy SMILES of hits.

Summary of changes in search.html:

  • Add 'Ask Questions' and contact info
  • Add 'Database Information' hyperlink
  • Pre-toggle Advanced Option

Summary of changes in smallworld-swagger.yaml:

  • Change API route to be just "/"

Install Prerequisite Packages

  1. Install Apache Maven
    • dnf install maven -y
  2. In your home directory, create a new directory to hold the files for the upcoming procedures
    • mkdir /mnt/nfs/home/jjg/smallworld_build_from_source
  3. Download these latest Smallworlds and store them in 'smallworld_build_from_source/'. Here's how to download Smallworld.
    • smallworld-java-5.5.tar.gz
    • smallworld-java-src-5.5.tar.gz
  4. Extract contents from the tar.gz files
    • tar -xvf smallworld-java-5.5.tar.gz
      tar -xvf smallworld-java-src-5.5.tar.gz
  5. Install Apache Maven Smallworld Dependencies
    • cd /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java/lib/
      bash mvn-install.sh

Customizing swsearch.js

Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js

Conditional Hyperlinks

  1. vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js
  2. Search for this
    • ?div.append("<b>" + id + "</b>");
  3. Replace the inside of the else statement with these lines of code
    •                 if (id.substring(0,6) == "ZINC00") {
                              div.append("<b><a target='_blank' href='https://zinc20.docking.org/substances/"+id+"'>" + id + "</a></b>");
                      } else if(id.substring(0,4) == "ZINC" && id.substring(4,6) != "00") {
                              div.append("<b><a target='_blank' href='https://cartblanche22.docking.org/substance/"+id+"'>" + id + "</a></b>");
                      } else {
                              div.append("<b>" + id + "</b>");
                      }

Copy SMILES of Hits

  1. Search for this
    ?var $props
  2. Add this line at the end of the appends
                                   .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
  3. It should look like this
            var $props = $('<div>').append('<b>SWIDX:</b> ' + data.anonIdx)
                                   .append('<br/><b>MW:</b> ' + data.mw)
                                   .append('<br/><b>MF:</b> ' + data.mf)
                                   .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
  4. Then add this function anywhere except inside another function's scope
    function copySmiles(smiles) {
            navigator.clipboard.writeText(smiles);
    }

Customizing search.html

Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html

Add 'Ask Questions' and 'Database Information'

  1. vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
  2. Search 'Scoring Methods'
    ?Scoring Methods
  3. At the end of that big block of 'div', add these:
    <div class="section-head">
            Ask Questions
            <table class="swopt searchasyoudraw" style="width: 100%">
                    <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
            </table>
    </div>
    <div class="section-head">
            Database Descriptions
            <table class="swopt searchasyoudraw" style="width: 100%">
                    <th style="text-align: left">
                           <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                    </th>
            </table>
    </div>
  4. It should look like this
                <div class="section-head">
                    Scoring Methods
                </div>
                <table id="swscore">
    
                </table>
            </div>
            <div class="section-head">
                    Ask Questions
                    <table class="swopt searchasyoudraw" style="width: 100%">
                            <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
                    </table>
            </div>
            <div class="section-head">
                    Database Descriptions
                    <table class="swopt searchasyoudraw" style="width: 100%">
                            <th style="text-align: left">
                                    <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                            </th>
                    </table>
            </div>

Pre-toggle Advanced Option

  1. Inside the script element, add this function
    window.onload = function() {
            $('#advPanel').css('display', $(this).prop('checked', true) && 'block'); update_visible_columns();
    }
  2. Then search for "update_visible_columns" and scroll till you find the input checkbox element
    ?update_visible_columns
  3. In the input box element, add the checked option. Should look like this
    <input name="optAdv" type="checkbox"
                               onchange="$('#advPanel').css('display', $(this).prop('checked') ? 'block' : 'none'); update_visible_columns();" checked>

Change API Route to "/"

  1. path smallworld-java-src/web/target/sw-ws-5.6.1/WEB-INF/static/swagger/smallworld-swagger.yaml

Compile Code To Make WAR File

  1. Build WAR File
    cd /nfs/home/jjg/smallworld_build_from_source/smallworld-java-src
    mvn install -Pbootable
  2. If it works, then the new WAR File is located at /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/target/sw-ws-5.5.war

Restarting Smallworld Instance(s)

Old Way

  1. Ssh to abacus and become root
  2. execute run_sw_on_reboot.sh to show restart all instances on the machine
    bash /root/run_arthors_on_reboot.sh
  3. execute start_smallworld to restart specific Smallworld instance. It will show you options to choose from.
    start_smallworld

New Way

  1. Ssh to abacus and become root. Then run
    systemctl <start|stop|restart> <sw|swp|swc|swbb|swcc>