GPFS

From Leo's Notes
Revision as of 17:47, 4 February 2020 by Leo (talk | contribs) (Created page with " == Usage == The {{code|gpfs*}} packages supplies the {{code|mm*}} binaries under the {{code|/usr/lpp/mmfs/bin/}} directory. The GUI utilities are at {{code|/usr/lpp/mmfs/gui...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
This page was last edited on 4 February 2020, at 17:47.

Usage

The gpfs* packages supplies the mm* binaries under the /usr/lpp/mmfs/bin/ directory.

The GUI utilities are at /usr/lpp/mmfs/gui/cli/.


Command Description
mmccr flist lists files stored in cluster configuration repository (CCR)
mmccr lsnodes Shows all quorum nodes
mmshutdown -a Run on ems1 to shut down IBM cluster
mmstartup -a Run on ems1 to start up the IBM cluster
mmhealth node show Shows the health of all nodes
mmlsfs all Shows all file system attributes on the GPFS cluster.
mmlssnapshot gpfs0/1 will show all snapshots for a particular file system.
mmlsmount all_local shows whether the file system on this node is being mounted by other nodes in the cluster.
/usr/lpp/mmfs/bin/mmlspdisk all --not-ok shows all failed disks.
mmedquota -j gpfs1:fileset used to change a quota of a fileset on the GPFS1 volume.
mmcrfileset gpfs1 morph --inode-space new --inode-limit 4194304:2097152 change inode quota



Tasks

Create Filesystem

mmcrfs gpfs1 -F vdisk.cfg -j scatter -T /gpfs/gpfs1 -B 16M --metadata-block-size 1M

Create a new GPFS filesystem on GPFS1, where vdisk.cfg contains the filesystem specifications. 

%vdisk: vdiskName=gssio1_Data_16M_2p_1 rg=gssio1 da=DA1 blocksize=16M size=700T raidCode=8+2p diskUsage=dataOnly failureGroup=30  pool=data
%vdisk: vdiskName=gssio1_MetaData_16M_2p_1 rg=gssio1 da=DA1 blocksize=1M size=7T raidCode=3WayReplication diskUsage=metadataOnly failureGroup=30  pool=system
# %nsd:gssio1_Data_16M_2p_1:::dataOnly:30:gssio1_Data_16M_2p_1:data
gssio1_Data_16M_2p_1:::dataOnly:30::data
# %nsd:gssio1_MetaData_16M_2p_1:::metadataOnly:30:gssio1_MetaData_16M_2p_1:system
gssio1_MetaData_16M_2p_1:::metadataOnly:30::system
%vdisk: vdiskName=gssio2_Data_16M_2p_1 rg=gssio2 da=DA1 blocksize=16M size=700T raidCode=8+2p diskUsage=dataOnly failureGroup=30  pool=data
%vdisk: vdiskName=gssio2_MetaData_16M_2p_1 rg=gssio2 da=DA1 blocksize=1M size=7T raidCode=3WayReplication diskUsage=metadataOnly failureGroup=30  pool=system
# %nsd:gssio2_Data_16M_2p_1:::dataOnly:30:gssio2_Data_16M_2p_1:data
gssio2_Data_16M_2p_1:::dataOnly:30::data
# %nsd:gssio2_MetaData_16M_2p_1:::metadataOnly:30:gssio2_MetaData_16M_2p_1:system
gssio2_MetaData_16M_2p_1:::metadataOnly:30::system

Node Management

Use mmlsnode to show all nodes that are part of a nodeset. 

[root@tsm /]# mmlsnode
GPFS nodeset    Node list
-------------   -------------------------------------------------------
   chgi-psc     essio1-ib essio2-ib ems1-ib crick-ib powerlc001-ib powerlc002-ib transmart-ib tsm-ib node001 node002 node003 node004 node005 node006 node007 node008 node009 node010 node011 node012 node013 node014 node015 node016 node017 node018 node019 node020 node021 node022 node023 node024 node025 node026 node027 node028 node029 node030 node031 node032 node035 node033 node036 node034 theia galaxy-dev-ib ebg01-ib snyder-ib

Additional information for each node can be found when listing cluster information with mmlscluster: 

[root@tsm /]# mmlscluster

GPFS cluster information
========================
  GPFS cluster name:         chgi-psc.gpfs.net
  GPFS cluster id:           2878855853427990921
  GPFS UID domain:           chgi-psc.gpfs.net
  Remote shell command:      sudo wrapper in use
  Remote file copy command:  sudo wrapper in use
  Repository type:           CCR

 Node  Daemon node name        IP address      Admin node name         Designation
-----------------------------------------------------------------------------------
   1   essio1-ib.gpfs.net      172.26.3.1      essio1-ib.gpfs.net      quorum-manager-perfmon
   2   essio2-ib.gpfs.net      172.26.3.2      essio2-ib.gpfs.net      quorum-manager-perfmon
   3   ems1-ib.gpfs.net        172.26.3.251    ems1-ib.gpfs.net        quorum-perfmon
   4   crick-ib.gpfs.net       172.26.4.252    crick-ib.gpfs.net       
   5   powerlc001-ib.gpfs.net  172.26.4.1      powerlc001-ib.gpfs.net  
   6   powerlc002-ib.gpfs.net  172.26.4.2      powerlc002-ib.gpfs.net  
   7   transmart-ib.gpfs.net   172.26.4.3      transmart-ib.gpfs.net   
   8   tsm-ib.gpfs.net         172.26.3.250    tsm-ib.gpfs.net         
   9   node001                 172.26.10.1     node001                 
  ...
  50   node034                 172.26.10.34    node034                 
  51   theia                   172.26.25.2     theia                   
  56   galaxy-dev-ib           172.26.10.116   galaxy-dev-ib           
  58   ebg01-ib                172.26.10.153   ebg01-ib                
  60   snyder-ib               172.26.10.105   snyder-ib

To add a new node: 

# mmaddnode -N damona-ib

To remove a node: 

# mmdelnode -N damona-ib
Verifying GPFS is stopped on all affected nodes ...
mmdelnode: [W] Could not cleanup the following unreached nodes:
damona-ib
mmdelnode: Command successfully completed
mmdelnode: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.


See Also