SmallWorld Documentation for Future Developer: Difference between revisions

From DISI
Jump to navigation Jump to search
No edit summary
 
(37 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==
= Introduction =
[https://www.nextmovesoftware.com/downloads/smallworld/documentation/SmallWorld.pdf Here is the link to Smallworld's manual]
[https://www.nextmovesoftware.com/downloads/smallworld/documentation/SmallWorld.pdf Here is the link to Smallworld's manual]
* Username: ucsf@nextmovesoftware.com
* Username: ucsf@nextmovesoftware.com
Line 10: Line 10:
Execute command '''start_smallworld''' to see options on which smallworld instance to start/restart.
Execute command '''start_smallworld''' to see options on which smallworld instance to start/restart.


== How to Download Smallworld ==
= How to Download Smallworld =
# Ssh to abacus and become root. Prepare directory
# Ssh to abacus and become root. Prepare directory
#: <source> mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version> </source>
#: <source> mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version> </source>
Line 22: Line 22:
#* <source> tar -xvf <file_name> </source>
#* <source> tar -xvf <file_name> </source>


== How to Launch Smallworld For The First Time ==
= How to Launch Smallworld For The First Time =
=== Prepare Files and Directories ===
== Prepare Files and Directories ==
# Ssh to abacus and become root
# Ssh to abacus and become root
# Open a port in the firewall
# Open a port in the firewall
Line 72: Line 72:
#* '''SearchTimeOut''' should be the only option you'll change in the future. It caps search time to lessen strain on machine.
#* '''SearchTimeOut''' should be the only option you'll change in the future. It caps search time to lessen strain on machine.
#Place any *.anon.maps you've already compiled for testing into the maps sub-directory.
#Place any *.anon.maps you've already compiled for testing into the maps sub-directory.
=== Start Smallworld Instance ===
== Start Smallworld Instance ==
# Go to smallworld extract directory
# Go to smallworld extract directory
#: <source>cd /export/db3/smallworld_extract</source>
#: <source>cd /export/db3/smallworld_extract</source>
Line 82: Line 82:
#: <source>java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number></source>
#: <source>java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number></source>


==Creating Maps on SmallWorld==
= How to Build Smallworld Maps/Databases =
See the corresponding wiki page:
Consolidate smiles into '''/export/db3/temp_mapping'''.
http://wiki.docking.org/index.php/Creating_Maps_on_SmallWorld


==Running SmallWorld from the Command Line==
Use the script located in that directory called '''smallworld_map_script'''.
We have been running SmallWorld on n-9-22 as www which uses csh as the default shell. 
There is a file in /mnt/nfs/soft/www/home/smallworld called run_sw which contains the following commands to set up the SmallWorld Command Line Interface (CLI)


Set the SWDIR environment variable so SmallWorld scripts can access the 12 TB index
Here is the content of the script:
    setenv SWDIR /srv/db4
<source>
#!/bin/csh -f
setenv SWDIR /export/db3/public_smallworld_4th_gen/


Set the path to the SmallWorld C++ Executables
alias sw 'time java -jar /opt/nextmove/smallworld-5.1/sw.jar'
    set path = ($path /opt/nextmove/bin)
Alias the keyword "sw" to access the Java CLI
    alias sw 'java -jar /opt/nextmove/sw.jar'


==Tips for Running SmallWorld on the Command Line==
set target = '*.smi'
Use sw help to get information about available commands
 
    sw help
foreach i ($target)
echo '(sw map' $i '-T tmp/ -o '${i}'.anon.map -u '${i}'.anon.unmapped) |& tee '${i}'_log'
(sw map $i -T tmp/ -o ${i}.anon.map -u ${i}.anon.unmapped) |& tee ${i}_log
echo 'sw blm -p 0.01 '${i}'.anon.map'
sw blm -p 0.01 ${i}.anon.map
end
</source>
== Command Details ==
'''setenv''' sets the environment where the anon master map is
 
'''java -jar /opt/nextmove/smallworld-5.1/sw.jar''' shortened to '''sw''' via alias
 
'''map''' will map the smiles based on the current anon master map.
*'''-T''' stores temp files so /tmp does not get overloaded
*'''-o''' output with custom name you like
*'''-u''' will collect the unmapped molecules to send to Nextmove or to use swextend
'''tee''' will log the progress of the process
 
== Splitting Databases ==
[[http://wiki.docking.org/index.php/Creating_Maps_on_SmallWorld| Follow this page if John has asked you to split the database into smaller slices.]]
 
= Active Smallworld Instances =
{| class="wikitable"
|+ Caption text
|-
! Type !! URL !! Machine:Port !! Directory
|-
| Public || http://sw.docking.org/ || 10.20.0.5:5020 || /nfs/db3/public_smallworld_4th_gen
|-
| Private || http://swp.docking.org/ || 10.20.0.5:8080 || /nfs/db3/private_smallworld_4th_gen
|-
| Super Private || http://swc.docking.org/ || 10.20.0.5:5099 || /nfs/db3/super_private_smallworld_4th_gen
|-
| SWBB || http://swbb.docking.org/ || 10.20.0.5:8081 || /nfs/db3/swbb_5th_gen
|-
| SWCC || http://swcc.docking.org/ || 10.20.0.5:8082 || /nfs/db3/swcc_5th_gen
|-
|}
 
= Customize Smallworld FrontEnd To Our Needs (Smallworld 5.5) =
'''These instructions only worked and compiled for me in the machine called epyc which is running Rocky 8 Linux operating system.'''
 
Summary of changes in swsearch.js:
* Conditional hyperlinks attached to smallworld hits. Leads to zinc20, cartblanche22, or nothing.
* Adding clipboard icon to copy SMILES of hits.
 
Summary of changes in search.html:
* Add 'Ask Questions' and contact info
* Add 'Database Information' hyperlink
* Pre-toggle Advanced Option
 
Summary of changes in smallworld-swagger.yaml:
* Change API route to be just "/"
 
== Install Prerequisite Packages ==
# Install Apache Maven
#* <source>dnf install maven -y</source>
# In your home directory, create a new directory to hold the files for the upcoming procedures
#* <source>mkdir /mnt/nfs/home/jjg/smallworld_build_from_source</source>
# Download these latest Smallworlds and store them in 'smallworld_build_from_source/'. [[SmallWorld_Documentation_for_Future_Developer#How_to_Download_Smallworld|Here's how to download Smallworld.]]
#* smallworld-java-5.5.tar.gz
#* smallworld-java-src-5.5.tar.gz
# Extract contents from the tar.gz files
#* <source>tar -xvf smallworld-java-5.5.tar.gz
tar -xvf smallworld-java-src-5.5.tar.gz</source>
# Install Apache Maven Smallworld Dependencies
#* <source>
cd /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java/lib/
bash mvn-install.sh
</source>
 
== Customizing swsearch.js ==
Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js
 
=== Conditional Hyperlinks ===
# vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js
# Search for this
#* <source>?div.append("<b>" + id + "</b>");</source>
# Replace the inside of the else statement with these lines of code
#* <source>
                if (id.substring(0,6) == "ZINC00") {
                        div.append("<b><a target='_blank' href='https://zinc20.docking.org/substances/"+id+"'>" + id + "</a></b>");
                } else if(id.substring(0,4) == "ZINC" && id.substring(4,6) != "00") {
                        div.append("<b><a target='_blank' href='https://cartblanche22.docking.org/substance/"+id+"'>" + id + "</a></b>");
                } else {
                        div.append("<b>" + id + "</b>");
                }
</source>
=== Copy SMILES of Hits ===
# Search for this
#: <source>?var $props</source>
# Add this line at the end of the appends
#: <source>
                              .append('<br/><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
</source>
# It should look like this
#: <source>
        var $props = $('<div>').append('<b>SWIDX:</b> ' + data.anonIdx)
                              .append('<br/><b>MW:</b> ' + data.mw)
                              .append('<br/><b>MF:</b> ' + data.mf)
                              .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
</source>
# Then add this function anywhere except inside another function's scope
#: <source>
function copySmiles(smiles) {
        navigator.clipboard.writeText(smiles);
}
</source>
 
== Customizing search.html ==
Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
 
=== Add 'Ask Questions' and 'Database Information' ===
# vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
# Search 'Scoring Methods'
#: <source>?Scoring Methods</source>
# At the end of that big block of 'div', add these:
#: <source>
<div class="section-head">
        Ask Questions
        <table class="swopt searchasyoudraw" style="width: 100%">
                <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
        </table>
</div>
<div class="section-head">
        Database Descriptions
        <table class="swopt searchasyoudraw" style="width: 100%">
                <th style="text-align: left">
                      <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                </th>
        </table>
</div>
</source>
# It should look like this
#: <source>
            <div class="section-head">
                Scoring Methods
            </div>
            <table id="swscore">
 
            </table>
        </div>
        <div class="section-head">
                Ask Questions
                <table class="swopt searchasyoudraw" style="width: 100%">
                        <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
                </table>
        </div>
        <div class="section-head">
                Database Descriptions
                <table class="swopt searchasyoudraw" style="width: 100%">
                        <th style="text-align: left">
                                <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                        </th>
                </table>
        </div>
</source>
=== Pre-toggle Advanced Option ===
# Inside the script element, add this function
#: <source>
window.onload = function() {
        $('#advPanel').css('display', $(this).prop('checked', true) && 'block'); update_visible_columns();
}
</source>
# Then search for "update_visible_columns" and scroll till you find the input checkbox element
#:<source>?update_visible_columns</source>
# In the input box element, add the checked option. Should look like this
#:<source>
<input name="optAdv" type="checkbox"
                          onchange="$('#advPanel').css('display', $(this).prop('checked') ? 'block' : 'none'); update_visible_columns();" checked>
</source>
 
=== Change API Route to "/" ===
# path smallworld-java-src/web/target/sw-ws-5.6.1/WEB-INF/static/swagger/smallworld-swagger.yaml
 
== Compile Code To Make WAR File ==
#Build WAR File
#: <source>
cd /nfs/home/jjg/smallworld_build_from_source/smallworld-java-src
mvn install -Pbootable
</source>
#If it works, then the new WAR File is located at '''/mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/target/sw-ws-5.5.war'''
 
= Restarting Smallworld Instance(s) =
=== Old Way ===
# Ssh to abacus and become root
# execute '''run_sw_on_reboot.sh''' to show restart all instances on the machine
#: <source>
bash /root/run_arthors_on_reboot.sh
</source>
# execute '''start_smallworld''' to restart specific Smallworld instance. It will show you options to choose from.
#: <source>
start_smallworld
</source>
=== New Way ===
# Ssh to abacus and become root. Then run
#: <source>
systemctl <start|stop|restart> <sw|swp|swc|swbb|swcc>
</source>

Latest revision as of 23:38, 1 May 2024

Introduction

Here is the link to Smallworld's manual

  • Username: ucsf@nextmovesoftware.com
  • Password: <Ask jjiteam@googlegroups.com>

All smallworld databases live in abacus' db3.

Smallworld frontend files are consolidated and located at /export/db3/smallworld_extract/.extract.

Execute command start_smallworld to see options on which smallworld instance to start/restart.

How to Download Smallworld

  1. Ssh to abacus and become root. Prepare directory
     mkdir /opt/nextmove/smallworld-<version> && cd /opt/nextmove/smallworld-<version>
  2. Download Software with this link
    • Username: ucsf@nextmovesoftware.com
    • Password: <Ask jjiteam@googlegroups.com>
  3. Go to releases. Look for smallworld-java-<version>.tar.gz and copy the link address.
  4. Download using wget
     wget --user ucsf@nextmovesoftware.com --password <Ask jjiteam@googlegroups.com> <link address>
  5. Decompress the file
    •  tar -xvf <file_name>

How to Launch Smallworld For The First Time

Prepare Files and Directories

  1. Ssh to abacus and become root
  2. Open a port in the firewall
    firewall-cmd --permanent --add-port=<port_number>/tcp 
    firewall-cmd --reload
  3. Go to smallworld databases directory
    cd /export/db3/
  4. Create a Smallworld directory and go to it
    mkdir <smallworld_dir> && cd <smallworld_dir>
  5. Inside that directory, create these
    • a symbolic link to the smallworld anon map
    ln -s /export/db2/smallworld_anon_21Q4 anon
    • a sub-directory called maps to store databases
    mkdir maps
    • a smallworld config file
    vim smallworld.cfg
    • Add this block of code inside the config file
    [WebApp]
    DepictionUrl=./depict/svg?w=%w&h=%h&smi=%s&qry=%q&cols=%c&cmap=%m
    ResolverUrl=./util/smi2mol?smi=%s
    SearchTimeOut=5000
    SearchAsYouDraw=true
    [Score]
    Name=Atom Alignment
    Description=Aligns and scores each hit relative to the query atom types, differences are categorised as MAJ, MIN, HYB, SUB 
    Class=com.nmsoftware.smallworld.score.SimpleAtomAlignment 
    EnabledByDefault=false
    MappingMode=EditMap
    [Score]
    Name=SMARTS Alignment
    Description=Treat input as SMARTS and align and score each hit relative to the query atom expressions in the input 
    Class=com.nmsoftware.smallworld.score.SmartsAtomAlignment 
    EnabledByDefault=false
    [Score]
    Name=ECFP4
    Description=Extended Connectivity Circular Fingerprint
    Class=com.nmsoftware.smallworld.score.EcfpTanimoto
    Radius=2
    EnabledByDefault=true
    [Score]
    Name=Daylight
    Description=Path based fingerprint
    Class=com.nmsoftware.smallworld.score.DaylightTanimoto
    EnabledByDefault=true
    • SearchTimeOut should be the only option you'll change in the future. It caps search time to lessen strain on machine.
  6. Place any *.anon.maps you've already compiled for testing into the maps sub-directory.

Start Smallworld Instance

  1. Go to smallworld extract directory
    cd /export/db3/smallworld_extract
  2. Create a screen
    screen -S <screen_name>
  3. Export the smallworld directory you've made
    export SWDIR=/export/db3/<smallworld_dir>
  4. Launch java
    java -jar /opt/nextmove/smallworld-<version>/sw-ws-<version>.jar --httpPort=<port_number>

How to Build Smallworld Maps/Databases

Consolidate smiles into /export/db3/temp_mapping.

Use the script located in that directory called smallworld_map_script.

Here is the content of the script:

#!/bin/csh -f
setenv SWDIR /export/db3/public_smallworld_4th_gen/

alias sw 'time java -jar /opt/nextmove/smallworld-5.1/sw.jar'

set target = '*.smi'

foreach i ($target)
	echo '(sw map' $i '-T tmp/ -o '${i}'.anon.map -u '${i}'.anon.unmapped) |& tee '${i}'_log'
	(sw map $i -T tmp/ -o ${i}.anon.map -u ${i}.anon.unmapped) |& tee ${i}_log
	echo 'sw blm -p 0.01 '${i}'.anon.map'
	sw blm -p 0.01 ${i}.anon.map
end

Command Details

setenv sets the environment where the anon master map is

java -jar /opt/nextmove/smallworld-5.1/sw.jar shortened to sw via alias

map will map the smiles based on the current anon master map.

  • -T stores temp files so /tmp does not get overloaded
  • -o output with custom name you like
  • -u will collect the unmapped molecules to send to Nextmove or to use swextend

tee will log the progress of the process

Splitting Databases

[Follow this page if John has asked you to split the database into smaller slices.]

Active Smallworld Instances

Caption text
Type URL Machine:Port Directory
Public http://sw.docking.org/ 10.20.0.5:5020 /nfs/db3/public_smallworld_4th_gen
Private http://swp.docking.org/ 10.20.0.5:8080 /nfs/db3/private_smallworld_4th_gen
Super Private http://swc.docking.org/ 10.20.0.5:5099 /nfs/db3/super_private_smallworld_4th_gen
SWBB http://swbb.docking.org/ 10.20.0.5:8081 /nfs/db3/swbb_5th_gen
SWCC http://swcc.docking.org/ 10.20.0.5:8082 /nfs/db3/swcc_5th_gen

Customize Smallworld FrontEnd To Our Needs (Smallworld 5.5)

These instructions only worked and compiled for me in the machine called epyc which is running Rocky 8 Linux operating system.

Summary of changes in swsearch.js:

  • Conditional hyperlinks attached to smallworld hits. Leads to zinc20, cartblanche22, or nothing.
  • Adding clipboard icon to copy SMILES of hits.

Summary of changes in search.html:

  • Add 'Ask Questions' and contact info
  • Add 'Database Information' hyperlink
  • Pre-toggle Advanced Option

Summary of changes in smallworld-swagger.yaml:

  • Change API route to be just "/"

Install Prerequisite Packages

  1. Install Apache Maven
    • dnf install maven -y
  2. In your home directory, create a new directory to hold the files for the upcoming procedures
    • mkdir /mnt/nfs/home/jjg/smallworld_build_from_source
  3. Download these latest Smallworlds and store them in 'smallworld_build_from_source/'. Here's how to download Smallworld.
    • smallworld-java-5.5.tar.gz
    • smallworld-java-src-5.5.tar.gz
  4. Extract contents from the tar.gz files
    • tar -xvf smallworld-java-5.5.tar.gz
      tar -xvf smallworld-java-src-5.5.tar.gz
  5. Install Apache Maven Smallworld Dependencies
    • cd /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java/lib/
      bash mvn-install.sh

Customizing swsearch.js

Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js

Conditional Hyperlinks

  1. vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/js/swsearch.js
  2. Search for this
    • ?div.append("<b>" + id + "</b>");
  3. Replace the inside of the else statement with these lines of code
    •                 if (id.substring(0,6) == "ZINC00") {
                              div.append("<b><a target='_blank' href='https://zinc20.docking.org/substances/"+id+"'>" + id + "</a></b>");
                      } else if(id.substring(0,4) == "ZINC" && id.substring(4,6) != "00") {
                              div.append("<b><a target='_blank' href='https://cartblanche22.docking.org/substance/"+id+"'>" + id + "</a></b>");
                      } else {
                              div.append("<b>" + id + "</b>");
                      }

Copy SMILES of Hits

  1. Search for this
    ?var $props
  2. Add this line at the end of the appends
                                   .append('<br/><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
  3. It should look like this
            var $props = $('<div>').append('<b>SWIDX:</b> ' + data.anonIdx)
                                   .append('<br/><b>MW:</b> ' + data.mw)
                                   .append('<br/><b>MF:</b> ' + data.mf)
                                   .append('<br/><b>Copy SMILES: </b><a href="#" onclick="copySmiles(\'' + get_smiles_parts(data.hitSmiles)[0] + '\');" ><i class="fa fa-clipboard" aria-hidden="true"></i></a>');
  4. Then add this function anywhere except inside another function's scope
    function copySmiles(smiles) {
            navigator.clipboard.writeText(smiles);
    }

Customizing search.html

Location: /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html

Add 'Ask Questions' and 'Database Information'

  1. vim /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/src/main/webapp/WEB-INF/static/search.html
  2. Search 'Scoring Methods'
    ?Scoring Methods
  3. At the end of that big block of 'div', add these:
    <div class="section-head">
            Ask Questions
            <table class="swopt searchasyoudraw" style="width: 100%">
                    <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
            </table>
    </div>
    <div class="section-head">
            Database Descriptions
            <table class="swopt searchasyoudraw" style="width: 100%">
                    <th style="text-align: left">
                           <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                    </th>
            </table>
    </div>
  4. It should look like this
                <div class="section-head">
                    Scoring Methods
                </div>
                <table id="swscore">
    
                </table>
            </div>
            <div class="section-head">
                    Ask Questions
                    <table class="swopt searchasyoudraw" style="width: 100%">
                            <th style="text-align: left">Email us: jjiteam@googlegroups.com</th>
                    </table>
            </div>
            <div class="section-head">
                    Database Descriptions
                    <table class="swopt searchasyoudraw" style="width: 100%">
                            <th style="text-align: left">
                                    <a href="https://wiki.docking.org/index.php/Smallworld_and_Arthor_Databases#Smallworld_Databases" target="_blank">Database Information Link</a>
                            </th>
                    </table>
            </div>

Pre-toggle Advanced Option

  1. Inside the script element, add this function
    window.onload = function() {
            $('#advPanel').css('display', $(this).prop('checked', true) && 'block'); update_visible_columns();
    }
  2. Then search for "update_visible_columns" and scroll till you find the input checkbox element
    ?update_visible_columns
  3. In the input box element, add the checked option. Should look like this
    <input name="optAdv" type="checkbox"
                               onchange="$('#advPanel').css('display', $(this).prop('checked') ? 'block' : 'none'); update_visible_columns();" checked>

Change API Route to "/"

  1. path smallworld-java-src/web/target/sw-ws-5.6.1/WEB-INF/static/swagger/smallworld-swagger.yaml

Compile Code To Make WAR File

  1. Build WAR File
    cd /nfs/home/jjg/smallworld_build_from_source/smallworld-java-src
    mvn install -Pbootable
  2. If it works, then the new WAR File is located at /mnt/nfs/home/jjg/smallworld_build_from_source/smallworld-java-src/web/target/sw-ws-5.5.war

Restarting Smallworld Instance(s)

Old Way

  1. Ssh to abacus and become root
  2. execute run_sw_on_reboot.sh to show restart all instances on the machine
    bash /root/run_arthors_on_reboot.sh
  3. execute start_smallworld to restart specific Smallworld instance. It will show you options to choose from.
    start_smallworld

New Way

  1. Ssh to abacus and become root. Then run
    systemctl <start|stop|restart> <sw|swp|swc|swbb|swcc>