DriversRecommendedOutdated drivers can make a good PC feel brokenScan driver issues before chasing fixes manually.Scan NowOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan Now×
Skip to content
World desk13 min

How to Add a Second SSH Key to a Linux VPS Without Removing the Old One

A practical guide to adding another SSH key for an existing Linux VPS user while preserving the current key, with safe installation methods and troubleshooting.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Yes, a single Linux account can accept multiple SSH keys. To add a second one, create a new key pair on your computer and append only its public-key line to that account’s ~/.ssh/authorized_keys file on the VPS. The existing line stays in place, so the original key continues to work.

Keep the private key on the client that generated it, protect it with a passphrase, and test the new login in a separate terminal before closing your current session. The safest installation method depends on whether you can still log in with a password, have an active SSH session, or only have access to your provider’s recovery console.

As an Amazon Associate I earn from qualifying purchases.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

How SSH decides which key is accepted

SSH public-key authentication uses a key pair. The private key remains on your local computer and proves possession of the key; the matching public key is stored on the server. When you connect, the SSH client offers a key and the server checks whether its public counterpart appears in the target account’s ~/.ssh/authorized_keys.

Each non-empty line in authorized_keys represents an accepted public key. Adding a second line adds another login option; it does not replace the first line. Ubuntu Server documentation describes this model and recommends Ed25519 keys because they are smaller and require less computation than older choices.

#1 Best Overall
Yubico - Security Key C NFC - Basic Compatibility - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-C or NFC, FIDO Certified
  • POWERFUL SECURITY KEY: The Security Key C NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
  • WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key C NFC secures 100 of your favorite accounts, including email, password managers, and more.
  • FAST & CONVENIENT LOGIN: Plug in your Security Key C NFC via USB-C and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
  • TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
  • BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.

The account matters. A key placed in /home/alex/.ssh/authorized_keys authorizes the user alex, not every account on the machine. If you normally connect as root, the relevant file is in root’s home directory, but a safer setup is usually to add the key to the intended non-root account and use administrative escalation after login.

What you need before starting

  • A local computer with an OpenSSH client and permission to create files in ~/.ssh.
  • The VPS hostname or IP address, the correct SSH username, and the correct SSH port if it is not the default.
  • One of three access paths: working password authentication, an existing SSH session, or a provider web/serial recovery console.
  • The current connection left open until the new key has been tested successfully.
  • A passphrase you can remember or store in an approved password manager for the new private key.

Do not upload, email, paste, or otherwise copy the private key to the VPS. Only the public file ending in .pub belongs in authorized_keys.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Choose the installation method

Method Access required Explicit key path Risk to the active account Best use
ssh-copy-id Password login still works Yes, with -i Low; it appends rather than replaces Fastest password-assisted setup
Pipe from an existing session An SSH connection that already works Yes, in the local cat command Low; keep the session open for recovery Servers that disable password authentication
Manual edit Shell access on the VPS Yes, when selecting the local public file Moderate; editing errors can break access When transfer utilities are unavailable
Provider recovery console Web or serial console and administrator access Yes, when you supply the public-key text Use carefully; you are editing the account directly No SSH login works

Choose the least invasive method that matches your access. Every method should append one complete public-key line, never overwrite the file. If a command asks which identity to use, specify the new public-key path explicitly so you do not accidentally install a different key.

Step 1: Generate a separate key pair on your computer

Use a distinct filename so the new key cannot overwrite an existing default key:

ssh-keygen -t ed25519 -f ~/.ssh/vps_second_ed25519

When prompted, enter a strong passphrase. Pressing Enter twice creates an unencrypted private key, which is less safe if the computer or its backups are compromised.

The command creates two files:

  • ~/.ssh/vps_second_ed25519 — the private key. Keep it on this client and restrict access to it.
  • ~/.ssh/vps_second_ed25519.pub — the public key that can be installed on the VPS.

Do not rename the files after installation unless you also update the command or SSH configuration that refers to them. You can inspect the public key before copying it:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
cat ~/.ssh/vps_second_ed25519.pub

A valid public-key entry is one complete line beginning with a key type such as ssh-ed25519, followed by encoded key material and optionally a comment. Copy the complete line without inserting line breaks.

The comment at the end is only a label for people reviewing the file. It does not grant access and does not need to match the VPS hostname. A useful label can identify the device or owner, making later revocation safer.

Step 2: Install it with ssh-copy-id

If password authentication is available, this is normally the simplest route:

ssh-copy-id -i ~/.ssh/vps_second_ed25519.pub [email protected]

Replace user and vps.example with the target account and host. The -i option is important because it selects the new public key instead of leaving the utility to guess which identity file to use.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
Yubico - YubiKey 5C NFC - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-C or NFC, FIDO Certified - Protect Your Online Accounts
  • POWERFUL SECURITY KEY: The YubiKey 5C NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
  • WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5C NFC secures 100+ of your favorite accounts, including email, password managers, and more
  • FAST & CONVENIENT LOGIN: Plug in your YubiKey 5C NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
  • MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
  • PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts

Enter the account password when prompted. The utility appends the selected public key to the remote user’s ~/.ssh/authorized_keys. Ubuntu’s ssh-copy-id manual says it can create the .ssh directory and the file when they do not yet exist.

Some minimal distributions do not include ssh-copy-id. Install the OpenSSH client utilities on your local operating system, or use the pipe method below. Do not respond to a missing utility by copying the private key.

If you use a nonstandard SSH port, include it in the command, for example ssh-copy-id -p 2222 -i ~/.ssh/vps_second_ed25519.pub [email protected]. A successful command only proves that the key was transferred; still perform a fresh login test with the matching private key.

Step 3: Append the key through an existing SSH session

This method is useful when the VPS accepts your current key but has password authentication disabled. Open a second local terminal and run:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
cat ~/.ssh/vps_second_ed25519.pub | ssh [email protected] 'umask 077; mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys'

The local cat reads only the public file. SSH transports that text to the server, where the remote shell creates the directory if necessary and appends the line. umask 077 prevents newly created SSH files from being readable or writable by other users.

Keep the original SSH window open while running this command. If the command reports an error, you can correct the file without losing your working access. If the account’s shell or home directory is unusual, log in normally first and use the manual procedure so you can confirm the exact path.

Run the command as the same account that should use the key. Connecting as an administrator and writing to that administrator’s home directory authorizes the wrong account, even if the key text itself is correct.

Step 4: Add the line manually on the VPS

When you already have shell access but cannot use ssh-copy-id, display the public key locally, copy its entire single line, and connect to the target account. Then run:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
umask 077
mkdir -p ~/.ssh
cat >> ~/.ssh/authorized_keys

Paste the public-key line, press Enter, and finish input with Ctrl-D. Alternatively, open ~/.ssh/authorized_keys in an editor and add the line at the end. Do not delete or alter the existing key line.

Check that the pasted entry remains one physical line. Wrapping that you see in a terminal or editor is harmless display wrapping; an actual newline in the middle of the key corrupts it. Avoid adding quotation marks around the line.

Rank #3
Yubico - YubiKey 5 NFC - Multi-Factor authentication (MFA) Security Key and passkey, Connect via USB-A or NFC, FIDO Certified - Protect Your Online Accounts
  • POWERFUL SECURITY KEY: The YubiKey 5 NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
  • WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 NFC secures 100+ of your favorite accounts, including email, password managers, and more
  • FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
  • MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
  • PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts

After editing, inspect the file and compare the new entry with the output of cat ~/.ssh/vps_second_ed25519.pub on the client. Check the beginning key type, the encoded middle section, and the optional comment. A truncated or partially selected line will be ignored even though the file itself opens normally.

Step 5: Use the provider recovery console when SSH is unavailable

If every SSH attempt fails, open the VPS provider’s web or serial recovery console. Log in as an administrator, switch to the intended user, and append the public key to that user’s ~/.ssh/authorized_keys. DigitalOcean documents this console approach, along with password-reset recovery, for servers that cannot be reached through SSH.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Confirm the user’s home directory before editing. A common recovery mistake is placing the line in the administrator’s file while trying to authorize a different account. After saving, verify ownership and permissions, exit the console, and test from your local computer with the explicit private-key path.

The console is a recovery path, not a reason to install the private key on the server. Take the public-key text from the trusted client, or use a secure method to display it, and paste only that text into the intended account’s file.

Verify the new key before closing the old connection

Start a fresh connection in a separate terminal and select the new private key explicitly:

ssh -i ~/.ssh/vps_second_ed25519 [email protected]

If the VPS listens on another port, include it:

ssh -p 2222 -i ~/.ssh/vps_second_ed25519 [email protected]

Test from a new terminal or, ideally, a second device. A test made inside the existing session does not prove that a new authentication attempt works. Confirm that you reach the expected account by checking the prompt or running whoami.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Do not close the original session until the explicit test succeeds. If the new login works, you can safely disconnect the old session and begin using the new key. If you are testing from the same computer, use IdentitiesOnly yes or temporarily move unrelated agent identities out of the way so the old key cannot silently satisfy the login.

Make repeat connections reliable with SSH configuration

Typing -i every time works, but a host entry makes the choice unambiguous. Create or edit ~/.ssh/config on your local computer:

Host my-vps-second-key
    HostName vps.example
    User user
    IdentityFile ~/.ssh/vps_second_ed25519
    IdentitiesOnly yes

Connect with:

ssh my-vps-second-key

IdentityFile points to the private key. IdentitiesOnly yes tells the client to use the configured identity rather than offering a collection of unrelated keys from an agent. This prevents a successful test from accidentally using the old key.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

If the VPS uses a nonstandard port, add a Port line to the same host block. Keep separate host aliases when different accounts or keys access the same server. Protect the configuration file and private key from other local users, and ensure the path is correct if you later move the key.

Rank #4
Yubico - Security Key NFC - Basic Compatibility - Multi-Factor Authentication (MFA) Key, Connect via USB-A or NFC, FIDO Certified
  • POWERFUL SECURITY KEY: The Security Key NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
  • WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key NFC secures 100 of your favorite accounts, including email, password managers, and more.
  • FAST & CONVENIENT LOGIN: Plug in your Security Key NFC via USB-A and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
  • TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
  • BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.

Set safe permissions and confirm ownership

SSH may reject a key when the account’s SSH directory or authorization file is writable by other users. From the target account, apply a strict baseline:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Ubuntu also documents chmod go-w .ssh/authorized_keys as a corrective command that removes group and other write permissions. Ensure the intended user owns both the .ssh directory and authorized_keys; an administrator can correct ownership through the recovery console if necessary.

Do not make the private key world-readable on the client. The local file should be accessible only to its owner, and a passphrase provides additional protection if the file is copied.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Permissions must be checked in the correct home directory. Using ~ as an administrator refers to the administrator’s home, not the target user’s. If you must repair another account as an administrator, use that account’s absolute path and then restore ownership to the account that will log in.

Troubleshoot a rejected second key

Symptom Likely cause What to check
Permission denied (publickey) Wrong username, wrong key, malformed line, or server-side permission problem Retry with ssh -vvv -i ~/.ssh/vps_second_ed25519 [email protected]; confirm the account, key path, complete public-key line, ownership, and modes.
The test succeeds but appears to use the old key The client selected another identity from its defaults or agent Use an explicit -i path and the IdentitiesOnly yes configuration setting.
ssh-copy-id cannot authenticate Password login is disabled or the supplied password is for another account Use the existing-session pipe, manual edit, or recovery console method.
The key was pasted but is ignored The public-key line was split, truncated, or pasted into another user’s file Compare the server line with cat ~/.ssh/vps_second_ed25519.pub; check the intended home directory and preserve the line as one entry.
Connection times out or is refused Wrong hostname, firewall path, or SSH port Verify the VPS address and configured port before diagnosing key authentication.
Permissions look correct but authentication still fails The server log contains the specific rejection reason On Ubuntu, watch the SSH service log with sudo journalctl -fu ssh.service while making one test connection.

Verbose output is safest when testing from the client because it shows which identity files are offered and whether the server accepts the key. Do not publish that output if it contains hostnames, usernames, or other environment details.

When reviewing verbose output, first confirm that the intended private-key file is offered. Then check whether the server accepts the corresponding public key. If no key is offered, the local path, permissions, agent, or configuration is wrong; if the key is offered and rejected, investigate the remote username, file contents, ownership, and SSH server policy.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Manage the old and new keys after the migration

Keep both keys during a transition

Leaving both public keys in authorized_keys is useful when moving between computers, rotating credentials, or handing access to a second administrator. Label entries with meaningful comments so you can identify them later; the comment does not determine authentication.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Remove a key when it is no longer trusted

After confirming that the replacement works, make a backup of authorized_keys, then remove only the obsolete key’s complete line. Do not remove every line unless you intentionally want to disable all key-based access for that account. A line deleted from the file can no longer authenticate that key, while the remaining lines continue to work.

Use one key per device or person

Separate keys let you revoke one laptop or user without disrupting everyone else. Sharing one private key makes attribution and revocation difficult. When a device is lost or a private key may have been exposed, remove its matching public line immediately and install a replacement generated on a trusted device.

Keep a record of which comment belongs to which device, but treat the comment as an aid for administration rather than a security control. The public key material, not the label, determines which private key can authenticate.

Best Value
FIDO2 U2F Security Key Passkey Two-Factor Authentication (2FA) USB Key PIN+Touch (Non-Biometric) USB-A Type TrustKey T110
  • Security Key : Protect your online accounts against unauthorized access by using FIDO2 and U2F authentication with T110. It's the world's most protective security key that works with windows, Mac OS, Linux as well as Chrome, Firefox, Edge and many other major browsers.
  • Certified with the new FIDO2 standard, T110 provides the benefit of fast login and strong protection against phishing, account takeover as well as many other online attactks.
  • Works with : Bank of America, Github, Google, Microsoft, DUO, Twitter, Facebook, Dropbox, Apple, ebay, BINANCE, mor and more.
  • Fits USB-A port : Insert the T110 security key into the USB-A port of each service and log in conveniently with one touch
  • For the driver download and user guide, please visit TrustKey Solutions Home support page.

Security practices that prevent common mistakes

  • Generate the new pair locally with Ed25519 and a strong passphrase.
  • Transfer only the .pub file or its text; never place the private key on the VPS.
  • Use an explicit filename and -i option so you know exactly which key is being tested.
  • Keep a working SSH session open until a new-session test succeeds.
  • Check the username, hostname, and port before changing server files.
  • Restrict ~/.ssh and authorized_keys permissions and verify ownership.
  • Back up the authorization file before removing an old entry.
  • Use the recovery console instead of repeatedly guessing when no SSH method works.
  • Install separate keys for separate people or devices instead of distributing one private key.
  • Remove a public-key line promptly when its private counterpart is lost, exposed, or no longer trusted.

Quick procedure checklist

  1. On the client, run ssh-keygen -t ed25519 -f ~/.ssh/vps_second_ed25519 and set a passphrase.
  2. Install ~/.ssh/vps_second_ed25519.pub with ssh-copy-id -i, the existing-session pipe, a manual edit, or the recovery console.
  3. Confirm that the new public-key line was appended to the intended user’s authorized_keys without deleting the old line.
  4. Test using ssh -i ~/.ssh/vps_second_ed25519 from a separate terminal.
  5. Add an IdentityFile and IdentitiesOnly yes host entry if you will connect regularly.
  6. Correct permissions or inspect ssh -vvv output and journalctl -fu ssh.service if the test fails.
  7. Close the old session only after the new key has authenticated successfully.

Frequently Asked Questions

Can two SSH keys work for the same Linux user?

Yes. Put each complete public-key entry on its own line in that user’s ~/.ssh/authorized_keys. SSH accepts any matching private key, so adding a line does not disable the existing key.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Will adding a second key delete the first one?

No. The append methods described here add text with >> or use ssh-copy-id, which appends the selected public key. Do not use a command or editor action that overwrites the file.

Which part of the key belongs on the VPS?

Only the public key, normally the file ending in .pub. The private file without that suffix must remain on the client and should be protected with a passphrase.

What if password authentication is disabled?

Use an SSH session that already works and append the public key through the pipe command, or edit the file manually from that session. If no SSH session works, use the provider’s web or serial recovery console.

How do I make sure I am testing the new key rather than the old one?

Specify the private key with ssh -i ~/.ssh/vps_second_ed25519. For repeat connections, configure that path with IdentityFile and set IdentitiesOnly yes so other agent identities are not silently used.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why does SSH say “Permission denied (publickey)” after I copied the key?

Check the username, hostname, port, complete single-line public key, ownership, and permissions. Run ssh -vvv on the client and watch sudo journalctl -fu ssh.service on Ubuntu while making one test connection.

Where should I add the key when I normally log in as root?

A key in root’s ~/.ssh/authorized_keys authorizes root only. If the key is for another account, place it in that account’s home directory and verify that account owns the file.

When can I remove the old key?

Keep the old session and key until a fresh connection using the new private key succeeds. Back up authorized_keys, then delete only the obsolete public-key line if it is no longer trusted or needed.

The Bottom Line

Generate a separate Ed25519 pair locally, append only its public line to the intended user’s authorized_keys, and leave the old line untouched. Use an explicit -i test from a new terminal, keep the current session open until it succeeds, and check ownership and permissions if authentication fails.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.