centos 7 – ExpLinux

Mysql resource is unable to start after failover due to permission change on ‘/var/lib/mysql’ directory.

explinux — Thu, 08 Apr 2021 11:05:00 +0000

If you are working on a cluster, after a sudden failover you face the issue that Mysql is unable to start.

In this article, we will show you how to solve the issue.

Environment

Red Hat Enterprise Linux Server 6 (with the High Availability and Resilient Storage Add Ons)
Red Hat Enterprise Linux Server 7
Red Hat Enterprise Linux Server 8
MySQL

Issue

Permission of mount point of MySQL data directory gets changed.

Mysql resource is unable to start in the other node after failover from the first node.


Sep  2 16:23:57 node A rgmanager[144383]: [fs] mounting /dev/dm-4 on /var/lib/mysql
Sep  2 16:23:57 node A kernel: EXT4-fs (dm-4): mounted filesystem with ordered data mode. Opts: 
Sep  2 16:23:57 node A rgmanager[144527]: [mysql] Checking Existence Of File /var/run/cluster/mysql/mysql:DB_name.pid [mysql:DB_name] > Failed 
Sep  2 16:23:57 node A rgmanager[144549]: [mysql] Monitoring Service mysql:DB_name > Service Is Not Running 
Sep  2 16:23:57 node A rgmanager[144571]: [mysql] Starting Service mysql:DB_name
Sep  2 16:24:27 node A rgmanager[145724]: [mysql] Starting Service mysql:DB_name > Failed - Timeout Error 
Sep  2 16:24:27 node A rgmanager[4142]: start on mysql "DB_name" returned 1 (generic error)

Resolution

Change the UID and GID of the MySQL user and keep it the same across all the cluster nodes.
Also, check that the UID and GID which you are going to assign for the MySQL user should not be possessed by any other user. If any then kindly change the UID and GID before changing the MySQL credentials.
Once after changing the credentials now try to failover the service from one node to another and the service should start without any issue.

Root Cause

The User and Group credentials of Mysql are not the same in all the cluster nodes.
```
On node A
# ls -ld /var/lib/mysql/
drwxr-xr-x 5 495 490 4096 Sep 11 00:21 /var/lib/mysql/
# grep -i mysql /etc/passwd
mysql:x:496:491:MySQL server:/var/lib/mysql:/bin/bash
```
On node B
# ls -ld /var/lib/mysql/
drwxr-xr-x 5 496 491 4096 Sep 11 00:22 /var/lib/mysql/
# grep -i mysql /etc/passwd
mysql:x:495:490:MySQL server:/var/lib/mysql:/bin/bash
As it can be seen from the above output that UID and GID of the MySQL user on the first node is 496 and 491 respectively. And the UID and GID of the MySQL user in the second node of the cluster is 495 and 490 respectively.
For mysql to access the data from /var/lib/mysql which is the default home directory, permission needs to be mysql:mysql.
Because of the different UID and GID of the mysql the user on both the nodes, when the cluster relocates the mysql resource from one node to other node it mounts the mysql data on the second node with the same UID and GID of the first node which does not match the mysql credentials on the second node due to which the mysql resource was not able to start on the second node.

Now you have successfully recovered from the issue. Please let us know if this helped you.

Grub2 files are missing from /boot. How to recover ?

explinux — Tue, 23 Feb 2021 02:13:00 +0000

Grub2 or Boot Repair

Due to some file corruption or mishandling we lost grub. This can cause the system to hang in the boot.

When you reboot the system it will not boot, So before rebooting the system check this file is in place or not. If you accidentally rebooted and the system is unable to boot then boot the server in rescue mode and follow the below steps.

Environment

Red Hat Enterprise Linux 7
CentOS 7
RHEL 8

Issue

Missing the ‘/boot’, grub folder, and grub.conf file which are essential to boot the system.
How to recover missing boot files ?

Resolution

Verify Boot directory contents are really missing or the /boot is not mounted properly :

#ls /boot/grub2/

Install grub2:

#grub2-install /dev/

Create grub.cfg file:

#grub2-mkconfig -o /boot/grub2/grub.cfg

Create device.map file if missing:

#vi /boot/grub2/device.map

(hd0)   /dev/

Reboot

Now the server or PC will boot properly. In this article, we have learned the way to recover grub and save it from reinstallation.

How to disable EEE feature on the nic on RHEL7-8/Centos7-8

explinux — Tue, 19 Jan 2021 07:25:00 +0000

Environment

Red Hat Enterprise Linux 7
Red Hat Enterprise Linux 8
CentOS 7
CentOS 8

Issue

Would like to disable the (Energy Efficient Ethernet) EEE feature of the NIC.

Resolution

To disable EEE use the ethtool command. In the example below, EEE is disabled on eth0:

# ethtool --set-eee eth0 eee off

To make this change persist across reboots, please add the ETHTOOL_OPTS parameter to the interface’s interface configuration file found in the /etc/sysconfig/network-scripts/ directory. For example:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:21:97:2E:EE:77
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=dhcp
ETHTOOL_OPTS="--set-eee ${DEVICE} eee off"

Diagnostic Steps

The ethtool command option –show-eee will display the current status of EEE:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# ethtool --show-eee eth0
EEE Settings for eth0:
    EEE status: enabled - inactive
    Tx LPI: 0 (us)
    Supported EEE link modes:  100baseT/Full 
                               1000baseT/Full 
    Advertised EEE link modes:  100baseT/Full 
                                1000baseT/Full 
    Link partner advertised EEE link modes:  Not reported

How to reset my MySQL database root password?

explinux — Sun, 17 Jan 2021 02:02:00 +0000

How to reset my MySQL database root password?

This is a very common issue if you are working in a big environment or a system where everything is automated. People always forgot their MySQL password. This article will help you recover your password without any data corruption.

Environment

Red Hat Enterprise Linux including:
- Red Hat Enterprise Linux 3
- Red Hat Enterprise Linux 4
- Red Hat Enterprise Linux 5
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 8
MySQL-server

Issue

Forgot the MySQL root password. And you want to know how to reset your password. How do I reset the mysql-server password?

# mysql -u root'
Access denied for user 'root'@'localhost' (using password: NO)'

Resolution

To reset the MySQL root password We have to follow the below procedure :

The first Step stop the MySQL service:

# service mysqld stop
Stopping MySQL:                                            [  OK  ]

Use the below command to start MySQL :
```
# /usr/bin/mysqld_safe --skip-grant-tables &
```
Note: On RHEL 3, mysqld_safe was called safe_mysqld:
```
# /usr/bin/safe_mysqld --skip-grant-tables &
```
Note: mysql_safe is a shell script this will only invoke mysqld but additionally traps any forceful terminations of the MySQL server and this will avoid any database corruption.

Change the password of the root user:

# mysql -u root mysql
mysql> UPDATE user SET Password=PASSWORD('new_password') WHERE user='root';
mysql> FLUSH PRIVILEGES;
mysql> exit;

Now Restart mysqld first use mysqladmin to ensure that the service shutdown successfully to avoid any other issue.
```
# mysqladmin -p shutdown
```
Now restart the MySQL service as per normal:
```
# service mysqld start 
```

Root Cause

The root password for mysqld was forgotten or lost.
You are a new admin and the old admin did not handover passwords.
Other team members reset the password and missed it.
Use of file to create a new server but MySQL password is not mentioned in doc.

How to set ulimit or change the default soft or hard limit for the number of user’s processes?

explinux — Fri, 11 Dec 2020 10:56:00 +0000

Environment

RHEL/CentOS 5
RHEL/CentOS 6
RHEL/CentOS 7
RHEL/CentOS 8

Issue

The default configuration file for ulimit is /etc/security/limits.d/90-nproc.conf for (RHEL5, RHEL6), /etc/security/limits.d/20-nproc.conf for (RHEL7) and for RHEL 8 we have to create file /etc/security/limits.d/20-nproc.conf specifies the default nproc limits as,:
```
*            soft    nproc     1024
root       soft    nproc     unlimited
```
Even when logged in as root, the ulimit shows a different value:
```
$ ulimit -u
1024
```
It is a bit confusing that it is not unlimited in this case?
Now we have a question in our mind that how to set or change the default soft or hard limit for the number of user’s processes?

Resolution

If we have to set the nproc limit to unlimited for all system , Need to change in the file /etc/security/limits.d/90-nproc.conf (RHEL5, RHEL6), /etc/security/limits.d/20-nproc.conf (RHEL7, RHEL8) should read:

For the root user:
```
root       hard    nproc     unlimited
```
For the ‘root’ user, both ‘soft’ and ‘hard’:
```
root       -    nproc     unlimited
```
or, for default domain again both ‘soft’ and ‘hard’ for example:
```
*          -    nproc     unlimited
```
The unlimited option applied to the soft value for a domain cannot go beyond its hard limit.
Re-login for changes to take effect:
```
$ ulimit -u
```
Another option to set the limits per session based is to specify it on the command line:
```
$ ulimit -u unlimited
```
The above command applies to both soft and hard limits.

Note:

1- The pam of RHEL5 and RHEL8 don’t provide any files under /etc/security/limits.d. If necessary, make the file.

2- RHEL8 does not have the file /etc/security/limits.d/20-nproc.conf, so if you want to limit nproc on RHEL8, you need to create a file like /etc/security/limits.d/20-nproc.conf in /etc/security/limits.d/.

Now you have successfully increased ulimit to unlimited. Please share this article if helpful

RHEL 7 Slow Login – Delay in SSH login

explinux — Sat, 14 Nov 2020 01:38:00 +0000

RHEL 7 Slow Login – Delay in SSH login

Why does it take a long time to get on login prompt after successful authentication with ssh with correct credentials?

ISSUE:

It takes more than 20 sec to login with ssh
Delay happens after successful login authentication
you find below error message in /var/log/secure

sshd[19312]: pam_systemd(sshd:session): Failed to create session: Connection timed out

RCA:

pam_systemd create CreateSession method in systemd-logind via DBus, and this means systemd-logind is unable to reply pam_systemd.
Sometimes the machine gets stuck or hangs during boot or due to high load.
‘systemd-logind’ following output error meaning, meaning that some directories have incorrect permission.

systemd-logind[681]: Failed to save session data /run/systemd/sessions/28: File exists

systemd-logind[681]: Failed to save user data /run/systemd/users/0: File exists

How To Fix This Login Delay in SSH:

Check the permission of the below directories this should be 755, If not then fix it.

/run/user
/run/systemd/users
/run/systemd/sessions

After the above status if the issue not fixes then we have to restart the below services step by step to resolve this issue.

1. You have to reload the system login based daemon

# systemctl daemon-reload

2. Check the health status of the inter-process communication daemon

# systemctl status dbus.service

# systemctl status polkit.service

3. If the dbus and polkit service is not running then you will start the first polkit service then we will start the dbus service. Please make sure this will start in the same order.

# systemctl start polkit.service

# systemctl start dbus.service

4. In the last step we have to restart the systemd-logind service

# systemctl restart systemd-logind

That’s it, now you have successfully fixed the issue. This is in the case when you have sufficient space in the root partition and load if optimal. If you have both issues then you have to fix them first.

Now you have added one more troubleshoot option in your skill.

Share the article if it helps you !!!

How to Use vi Editor in Linux and vi Editor Cheat-Sheet with 38 Shortcut

explinux — Sun, 05 Jul 2020 04:30:00 +0000

How to Use vi Editor in Linux and vi Editor Command in Linux

vi Editor :

. vi is the symbolic link of vim and this default editor comes with Linux based OS. This is very popular among sysadmins. This a very simple and powerful editor. You can use it to make your rivels list, a letter to your valentine, script, code, and edit any source code. This is your best friend in the Linux system.

vi Editor modes:

vi Editor has 3 modes –

1- Command mode :

This is the default mode when we enter in vi editor. In this mode almost every key n keyboard has a function, so use it very carefully.

Command mode when enter file using vi

Command mode when press esc key from any other mode

2- Insert Mode :

You can enter this mode by pressing the “i” key on the keyboard. In this mode, you can write anything your code, list, or edit file.

3- Visual mode or GUI mode :

This is not available in all distro and less used feature of vi editor. You can enter this mode by pressing the “v” key on the keyboard from command mode.

Note: By default vi editor mode is Command mode. You can enter any mode from command mode only and escape from that mode to command mode by pressing the “esc” key on the keyboard.

These are the modes of vi editor in Linux. Below you will see some vi editor cheat sheet which you can use in your sysadmin work and give you more understanding of vi editor commands in Linux.

vi Editor Basic Commands :

In this section, we will talk about the basic command of vi editor which has to know by every Linux user for use of the basic function of vi editor.

Create New File or Enter Any Existing File :

To enter any file use command vi . For example, see below

# vi explinux

you will enter in command mode as default by this command.

Write or edit in vi editors:

Press lowercase i to enter in edit mode. Now you can write anything or delete and navigate by keyboard navigation arrow key.

vi Editor Save and Exit :

When you finished your editing you have to come in command mode for exit or save the file. Please find below the steps to save and exit from the file.

Press esc key on the keyboard to change command mode from insert mode.
write “:wq! ” in lowercase. Press enter key to execute the command to save and exit from file. Please check below pic to understand more.

w – stands for save file

q – stands for quit from file

! – do not ask anything just run my command

vi Editor Cheat Sheet :

Now you have learned how to use vi editor and basic commands of vi editor which every Linux user should have known. Now we will talk about some advance command which should be known by every Linux sysadmin or any advance user.

These commands will run only in command mode as mentioned earlier. By default vi is in command mode if you are in another mode press esc key to enter in command mode.

Linux is a case sensitive so be careful about it.

List of Commands :

Open or edit any file :

# vi

Enter to Insert mode

Enter in Command Mode:

esc key on the keyboard

Save the file and Continue Editing:

Save and quit vi Editor :

:wq!

Quit from file and do not write change :

:q!

Yank :

Copy or yank line :

Copy X number of lines :

Xyy

for example 3 lines – 3yy

Paste Yanked File Below the Cursor Line :

Paste Yanked File Above the Cursor Line :

P or shift+p

This is the yank function that can be used to reduce time. This is a very useful command.

Create a New Line below the current line of Cursor :

Create a New Line Above the Current Cursor Line:

O or shift+o

Go To End of Line and Switch To Insert Mode for Write:

A or shift+a

Switch To Insert Mode At Start Of Line:

Switch to Insert Mode at the Beginning of Line :

I or shift+ i

Move to Begin of the Word :

Move to End of the Word:

Go to the Last Line of Open file :

G or shift+g

Go to the X Number Line of Open file :

for example, we have to go on 3rd line: 3G

Go to the X Number Line of Open file :

Delete a Single Letter or Character:

Delete a Single Word in Open File:

Delete X Number of Words in Open File :

Xdw

for example, delete 3 words: 3dw

Change Any Word in vi Editor :

Delete an Entire Current Line :

Delete X Lines in One Command :

Xdd

For example, we have to delete 3 lines then: 3dd

Change or Replace Any Character or Letter :

Overwrite All Characters form onwards Cursor Position:

R or shift+r

Get the Current Line’s line Number in Open File:

:num and press enter

Get All Lines’s Line Number in Open File:

:set nu and press enter key

Undo Last Change in vi Editor :

Undo Last Change in entire Line vi Editor :

U or shift+g

Search in vi Editor :

press n for the next result in an open file in vi editor .

Navigation In vi Editor :

Move left :

Move Down:

Move up :

Move right :

At this point, you are familiar with vi editor commands. Practice more to become pro at vi editor.

Summary :

The vi editor is most popular and distributed in every Linux distribution to edit almost all types of files.
There are 3 modes of vi editor
In this article, we covered almost all commands of vi editor

Share Article if you like it or report any bug in comments

How to Check and Disable Selinux in Centos7/8, RHEL 7/8

explinux — Tue, 30 Jun 2020 03:00:00 +0000

How to check and disable Selinux in Centos7/8, RHEL 7/8

SELinux – Security-Enhanced Linux

It is a security architecture that gives you more control over the system that others can access. It was integrated into Linux in 2003. SELinux is enabled by default and you can check SELinux status with the command sestaus. SELinux usage on AVC – access vector cache. You can configure SELinux for any subject or object.

Here in this article, we will talk about how to disable SELinux. As recommended do not disable it in the public domain if you do not have a firewall at the network level.

There are three modes of SELinux :

# enforcing stand for – SELinux security policy is enforced.

# permissive stands for – SELinux prints warnings instead of enforcing.

# disabled stand for – No SELinux policy is loaded.

How to disable the firewall in RHEL 7/8 / Centos 7/8

What is SELinux?

How To Disable SELinux

Step 1- Check the Status with the below command

#  sestatus

Now you can see in the above pic SELinux is in enforcing status. This means SELinux is active.

Step 2- Disable SELinux from the below command and check the status

# setenforce 0

This will disable SELinux temporarily. For permanent you need to do an entry in the configuration file.

Step 3- Make changes in the configuration file for permanent SELinux disable

There are two configuration files that we have to change to disable or permissive as per our requirement.

File -1 – Change by the below command

# vi /etc/sysconfig/selinux

Change permissive to disabled on SELINUX= *. This will disable SELinux permanently.

File -2 – Change by the below command

# vi /etc/selinux/config

Do the same as done in file -1. Save both files by (pressing the esc key):wq! Press enter.

4- Check now it will show disables in the config file.

# sestatus

Now at this point, you have successfully disabled SELinux and now you can check your services that are impacted by SELinux policy. Thanks, you can share this with your colleagues and request new articles in the comment section.

How to Use and Enable/Disable Firewall in Centos7/8, RHEL 7/8

explinux — Sun, 28 Jun 2020 03:00:00 +0000

How to Check and Disable Firewall in Centos7/8, RHEL 7/8

As we know the firewall is very important for our system or server security. In some cases where you have an external firewall in your network or you are using a different firewall, you need to disable the firewall. Few sysadmins disable the firewall for any application too but we recommend that for a case of application configure the firewall, do not disable it.

In this article, we will see how to check and disable the firewall in Linux.

Disable Firewall :

Step -1 Check firewall status with the below command

#  systemctl status firewalld

This will show the current status of the firewall. Now it is on active showing.

Step -2 Disable the firewall with below command

#  systemctl stop firewalld

This command stops the firewall temporarily but if you reboot or reload it will be active again.

Step – 3 Permanent disable it with the below command

#  systemctl disable firewalld

This will disable the firewall permanently. After this, you can manually start firewall.

Step -4 Check firewall status with the below command

#  systemctl status firewalld

Now you have successfully disabled the firewall of your Linux machine.

In case you want to Enable Firewall again. Follow the below procedure :

Enable Firewall

Step -1 Start firewall with below command

#  systemctl start firewalld

This will start firewall temporarily if it is disabled using disable command

Step -2 Enable the firewall permanently with the below command

 #  systemctl enable  firewalld

This will enable the firewall permanently.

Add a Port or Service in the Firewall

# firewall-cmd --zone=public --add-port=8080/tcp --permanent
# firewall-cmd --zone=public --add-service=tomcat --permanent
# firewall-cmd --reload

This will add ports and services to firewall

At this point, you know how to disable and enable the firewall on RHEL 7/8 and Centos 7/8. Share this if it is helpful.

How to Install Google Chrome 83 On Rhel 7 / Centos 7 And Dependency Resolution – 2021 Updated

explinux — Sun, 24 May 2020 18:07:00 +0000

How to Install Google Chrome 83 On Rhel 7 / Centos 7 And Dependency Resolution – 2021 Updated

This is an updated tutorial to install Google Chrome on Rhel 7 /Centos. Many of us facing dependencies issues in installing Google Chrome.

Google Chrome is a web browser from Google Chrome available free to use. Chrome is the most popular browser now.

In This Tutorial We Will learn:

How to install Google Chrome in RHEL7 /CENTOS 7
Resolve dependencies like glibc-common, etc
resolve the issue of ” –skip-broken to work around the problem”, “rpm -Va –nofiles –nodigest”
Create a local repo with packages

Our Environment :

RHEL 7.7 but this can work any version of RHEL / CENTOS 7. Rhel 7 yum repository configured.

1- Configure Google Chrome Repository

Create a file name googlechrome.repo in yum.repos.d directory. by the following command :

# vi /etc/yum.repos.d/googlechrome.repo

Do below line entry in the file and save it by :wq!

[google-chrome]

name=google-chrome

baseurl=http://dl.google.com/linux/chrome/rpm/stable/x86_64

enabled=1

gpgcheck=1

gpgkey=https://dl.google.com/linux/linux_signing_key.pub

Now we have 2 Repository

We will face issue of dependencies if we install google chrome from yum. So we will create a new repo with dependencies.

2- Configure dependencies Repository :

Create a directory extrarepo

# mkdir /extrarepo

Download below packages from https://pkgs.org/ or RedHat Package download page if you have a subscription. We have made simple for you , you can download from the below link also.

Chrome Dependency Package tar file use 7zip on windows or tar -xzvf filename in Linux to extract

glibc-2.30-alt1.x86_64.rpm

glibc-core-2.30-alt1.x86_64.rpm

libwayland-client-1.15.0-1.el7.x86_64.rpm

libX11-1.6.7-2.el7.x86_64.rpm

libX11-common-1.6.7-2.el7.noarch.rpm

libXau-1.0.8-2.1.el7.x86_64.rpm

libxcb-1.13-1.el7.x86_64.rpm

libXext-1.3.3-3.el7.x86_64.rpm

vulkan-1.1.97.0-1.el7.x86_64.rpm

vulkan-devel-1.1.97.0-1.el7.x86_64.rpm

vulkan-filesystem-1.1.97.0-1.el7.noarch.rpm

Use any tool, like WinSCP , Filezilla or you can directly download using utility wget to server. Here we have used WinSCP.

Install createrepo and run command createrepo /extrarepo . This will create repodata for packages.

# yum install -y createrepo
# createrepo /extrarepo/

Now create another file in /etc/yum,repos.d/Requiredpkg.repo

# vi /etc/yum.repos.d/Requiredpkg.repo

Do below lines in file and save from :wq!

[Required-Pkg]

name=Google Chrome required Pkg

baseurl=file:///extrarepo

enable=1

gpgcheck=0

Now we have 3 repositories and we are ready to install google chrome.

3- Install google chrome :

# yum install google-chrome-stable -y

This will install google chrome without any error

Check google chrome version by below command :

# google-chrome –version

Start google chrome by command :

# google-chrome &

Now at this point you have installed google chrome. Let us know if you face any issues. You can Dm also on our Insta page or use #explinux to ask questions.