Search my site
Twitter
Flickr
Archives

Blog Posts

Entries in linux (6)

Sunday
Nov282010

Fix Command Line Mistakes With Carat (^) Symbol Substitution

Open the Tip Jar Creative Commons License photo credit: usnationalarchives

Its time to open the tip jar again. The command line is incredibly powerful but requires perfect typing of commands and paths. I did not know how to fix a command until I read the lifehacker article. It seems every time I consider myself an expert I see a simple tip that *quashes* my ego back to novice status.

The Carat (^) symbol in many terminal or command shell environments is a simple text substitution command. It follows an idea of '^stringtorelpace^replacementstring'. This is great for fixing errors or substituting commands.

Fix a typo

mb2:~ mark$ la -lh /usr/local/nagios/etc/nagios.cfg
-bash: la: command not found
mb2:~ mark$ ^la^ls

This corrects the mistyped 'ls' command.

Change a command

Another interesting use is to change the command used. For example you may want to first list 'ls' the file to see if it exists, and if not, create it with 'nano'.

mb2:~ mark$ ls /usr/local/nagios/etc/nagios.cfg
ls: /usr/local/nagios/etc/nagios.cfg: No such file or directory
mb2:~ mark$ ^ls^nano

There are many time saving tricks that can be used with this simple command. Check out the source articles for more information.

Wednesday
May062009

chmod recursively on directories or files using find

Sometimes its the little things that annoy us so much on the Unix command line. One big question for me was... How to do you differentiate between directories and files when recursively chmod-ing?

The answer is simple. But of course there are different ways to do the same thing.

Using the find command

  find -name '*' -type d -exec chmod 0755 {} \;
NOTE: the -name '*' parameter is used to keep from modifying the present working directory or '.' directory.
  find . -type f -exec chmod 0644 {} \;

Using chmod with capital X

The capital X will cause directories and files that are executable (for user and group) to be set as executable. Other files will not be set as executable.

  chmod -R ug+rX .

Other uses of the find command

Modifying specific file types:

  find -name '*.pdf' -exec chmod 0755 {} \;
NOTE: you can insert any command in after the -exec but before the {} (chmod 0775) such as 'chown'.

My common usage

I often setup new Wordpress installations. I like to get ownership and permission sorted out quickly. I'll use this as my example.

In your Wordpress root directory:

  chown your-username.www-data * -R
  find -name '*' -type d -exec chmod 2750 {} \;
  find . -type f -exec chmod 2640 {} \;
NOTE: the 2750 and 2640 sets a bit so that all directories that are created by the www-data user in my case will have the same permissions and ownership as the the other files. Otherwise the www-data user may create files that are owned by itself and the your-username may not be able to modify them easily.

Now change so that the wp-content/{uploads,plugins,themes} directories are writable to the www-data group. This is so that the web server can upload photos and auto-update plugins/themes.

  cd wp-content
  mkdir uploads
  chown your-username.www-data uploads
  find -name '*' -type d -exec chmod 2770 {} \;
  find . -type f -exec chmod 2660 {} \;
Thursday
Apr162009

Scripting RSS Torrent Downloads

the brazilian article that used the RH pix Creative Commons License photo credit: irina slutsky

I had some interesting requirements for my Bittorrent needs. First I wanted to move everything to my linux mediaserver/fileserver. So I needed a way to work with bittorrent files without traditional access to the machine. eg. No screen or windowing system. This led me to TorrentFlux-b4rt, which is a php based web interface to Transmission client meant to be run on apache servers. Second I wanted to ability to download torrents automatically from RSS sources using scripting. This is where the original TorrentFlux had failed me and I went searching. The following is a gathering of scripts I have found and tested. Most scripts require that you have access to Cron scheduling and the python scripting language. And, of course, a bittorrent client that scans a directory looking for torrent files. I am using these scripts in conjunction with rTorrent (a commandline bittorrent client) as well as TorrentFlux-b4rt on various servers.

FlexGet

FlexGet is a program aimed to automate downloading content (torrents, podcasts, etc.) from various sources like RSS-feeds, html-pages, various sites and more. It's most often used to download torrent-files from RSS-feeds and works very well in that environment, but there are additional modules for other kind of situations as well. FlexGet is extremely useful in conjunction with applications and clients which have watch directory support. - http://flexget.com/

I've been using flexget over the last few weeks and it is my top pick for script to use.

rsstorrent

rsstorrent is a very simple RSS torrent downloading script. It does as advertised and is still being developed. Though it lacks the nice documentation of FlexGet.

tvrss member script

This is the script that I had been using for about a year to automate the torrent downloading. It scans the tvrss.net website using an easy to understand configuration file. It worked like a charm for me but I found a new charm with more sparkle.

TED - Torrent Episode Downloader

This is a different way to download torrent episodes. This is not a script. It is a fully functional cross-platform program dedicated RSS Torrents. It is way to heavy for my use. But interesting none the less.

ted can find episodes of any TV show you like to watch. Just add your favorite shows to ted and he will search for the newest episodes and downloads them for you. ted uses bittorrent and RSS technology to get you the newest episodes as fast as possible! http://www.ted.nu/
Friday
Mar272009

Howto install Torrentflux-b4rt on Ubuntu

What is Torrentflux-b4rt?

What is Torrentflux-b4rt you say? Torrentflux-b4rt is a sophisticated web-based PHP BitTorrent client based off the original Torrentflux. It allows you to use an old linux server as your happily obedient bittorrent client , while you gallivant around town feeding it torrents. Its greatest feature is freeing you from being constantly connected to the internet while downloading.

This guide is intended to be followed by a linux user confident with the command line and in full control of a server they wish to employ the use of TorrentFlux with. Good luck.

Base System

I am going to describe how to setup a basic server with these sources:

  • Ubuntu Server 8.04 LTS (hardy)
  • basic apt sources. (hardy main, hardy-updates main, hardy-securrity main)

Installation

Basic Requirements

  • LAMP Stack
  • Python
  • Perl
  • A custom transmission-cli bittorrent client interface

LAMP ( Linux, Apache, MySQL, PHP )

A cleaver guide has already been written Installing LAMP On Ubuntu For Newbies.

PHPMyAdmin

This gives you a web interface into the MySql server. Greatly simplifies life with MySql.

sudo apt-get install phpmyadmin

Install extra packaged needed for Torrentflux

sudo apt-get install zip unzip unrar php5-cli php5-gd perl python libxml-dom-perl libxml-simple-perl libthreads-shared-perl libdigest-sha1-perl libhtml-parser-perl transmission-cli
These are what I found missing from my install. You may come up with some missing packages 'apt-get' to install packages and google to find the package names.

Install Torrentflux-b4rt

Download the latest package Torrentflux-b4rt package

From your home directory:

cd ~
wget http://prdownload.berlios.de/tf-b4rt/torrentflux-b4rt_1.0-beta2.tar.bz2
tar xjvf torrentflux-b4rt_1.0-beta2.tar.bz2

Create the custom Transmission-cli client

Download the Transmission 1.06 Source

cd ~/torrentflux-b4rt_1.0-beta2/clients
wget http://download.m0k.org/transmission/files/transmission-1.06.tar.bz2
tar jxvf transmission-1.06.tar.bz2
tar jxvf transmission-1.06.tar.bz2
tar jxvf Transmission-1.06_tfCLI-svn3356.tar.bz2
cp Transmission-1.06_tfCLI-svn3356/cli/transmissioncli.c transmission-1.06/cli/transmissioncli.c
This overwrites the original source code for the Transmission-cli a new version extended for the Torrentflux-b4rt interface.

Install from Transmission from source

./configure
make
make install
This should install the transmission-cli binary in /usr/local/bin/transmission-cli

Copy the Torrentflux-b4rt html directory to a web visible location

cp ~/torrentflux-b4rt_1.0-beta2/html/ /var/www/torrentflux -R

Create the Torrentflux database and user using PHPMyAdmin

Log Into PHPMyAdmin

Log into your phpmyadmin interface using your MySQL root user/password by going to:

http://yourserverip/phpmyadmin

Create the User and Database

Steps to follow from the phpmyadmin frontpage:

  • Click the 'Privileges' link from the home page.
  • Then click on the "Add a new User" link.
  • Create a new user named 'torrentflux', set a strong password, select "Create database with same name and grant all privileges", click go.

Run the TorrentFlux-b4rt setup.php script

Point your browser to:

http://yourserverip/torrentflux/setup.php
Follow the directions onscreen to finish the setup. Give it your MySQL username, password, and database name. It should solve the rest.

Additional Resources

In case you get stuck or would like to more information check out these pages:

Thursday
Mar192009

DenyHosts smart defense for your SSH server

DenyHosts smart SSH Server Protection

DenyHosts is a Python based security tool for SSH servers. DenyHosts is a Python based security tool for SSH servers.

DenyHosts is a python script that is run on any Linux or BSD based system to help block SSH based attacks. It works to prevent both "dictionary based" and "brute force" attacks. Also provided is a system to synchronize block lists between other users of DenyHosts.

It works by scanning the SSHD log files (/var/log/auth.log or /var/log/secure) and discovering failed login attempts. Attacks are triggered by number of failed attempts or invalid username login attempts. After it discovers an attack it inserts the IP address into the /etc/hosts.deny file which will block that IP from connecting to your system in the future. Check out the features page for a full set of features. I started using DenyHosts after I noticed a surprising number of failed login attempts in my sshd logs. After I installed DenyHosts it discovered over 50 IPs that were attempting to gain access to my system and started blocking them. I quickly discovered the synchronization features and have not looked back. Now I maintain a blocklist near 9000 IPs long.

Base System

I am going to describe how to setup a basic server firewall with this base system as the example:

  • Ubuntu Server 8.04 LTS (hardy)
  • basic apt sources. (hardy main, hardy-updates main, hardy-security main)

Installation

  sudo apt-get install denyhosts

Configuration

Modify the configuration file /etc/denyhosts.conf to include these settings.

  SECURE_LOG = /var/log/auth.log
  HOSTS_DENY = /etc/hosts.deny
  PURGE_DENY = 			# When set to blank system will never purge block list.
  BLOCK_SERVICE  = sshd		# When set to SSHD the ssh server will be blocked,
  				# when set to ALL the IP will be blocked for all services.
  DENY_THRESHOLD_INVALID = 5
  DENY_THRESHOLD_VALID = 10
  DENY_THRESHOLD_ROOT = 5
  DENY_THRESHOLD_RESTRICTED = 5
  WORK_DIR = /var/lib/denyhosts
  SUSPICIOUS_LOGIN_REPORT_ALLOWED_HOSTS=YES
  HOSTNAME_LOOKUP=YES		# This will do a hostname lookup.  Set to NO for improved performance.
  LOCK_FILE = /var/run/denyhosts.pid

         ############ THESE SETTINGS ARE OPTIONAL ############
  ADMIN_EMAIL = 			# Leave Blank or use your email address for reports on blocked IPs.
  SMTP_HOST = localhost           # localhost if you are running your own email server or set to an external server
  SMTP_PORT = 25
  SMTP_FROM = DenyHosts
  SMTP_SUBJECT = DenyHosts Report
  SMTP_DATE_FORMAT = %a, %d %b %Y %H:%M:%S %z
  AGE_RESET_VALID=5d
  AGE_RESET_ROOT=25d
  AGE_RESET_RESTRICTED=25d
  AGE_RESET_INVALID=10d

     ######### THESE SETTINGS ARE SPECIFIC TO DAEMON MODE  ##########
  DAEMON_LOG = /var/log/denyhosts
  DAEMON_SLEEP = 30s
  DAEMON_PURGE = 1h

     #########   THESE SETTINGS ARE SPECIFIC TO     ##########
     #########       DAEMON SYNCHRONIZATION         ##########
  SYNC_SERVER = http://xmlrpc.denyhosts.net:9911
  SYNC_INTERVAL = 1h
  SYNC_UPLOAD = yes
  SYNC_DOWNLOAD = yes
  SYNC_DOWNLOAD_THRESHOLD = 3
Sync Server setup is optional. If you only want to block attempts on your server comment out the 'SYNC_SERVER' line.

Start the Service

  sudo /etc/init.d/denyhosts restart

Other Resources