Latest Diaspora* update was a lot of work

So I’ve had some issues with Diaspora*, but the main one is inter-pod communication which doesn’t seem to work back to the main joindiaspora.com pod (and some others). I had neglected updating for a little while and thought I should get to it, and see if this issue was resolved.

**Caution – the rest of this discussion is fairly technical **

I had some trouble with the oath2 ruby gem for some time now as well, and during the update process I kept receiving errors about the .gemspec file format. I went through the usual update process to see if there would be an update to oath2 amongst others. So I used my usual 3 update commands:

git pull origin master
DB="mysql" bundle install --without development,test
RAILS_ENV="production" DB="mysql" bundle exec rake db:migrate

to find that it was more upset than usual about the update. The last command didn’t want to run at all, and there were far too many errors to see what was going on.

The easiest solution to this I thought would be to clean up the Ruby gems by updating everything and seeing what happened. To do a more thorough udpate of the gems, I ran

bundle update

which resulted in:

WARNING:  Invalid .gemspec format in '/usr/lib/ruby/gems/1.8/specifications/oaut
h2-provider-0.0.19.gemspec'
Gems included by the bundle:
Could not find oauth2-provider-0.0.19 in any of the sources

So I tried

gem update

and that’s when it all got interesting. Apparently I had run out of disk quote. Disk quota? As root? I check quota, sure enough, no quota applied. I ran a df to find I was using only 23% of disk space. So I tracked it down in the end – out of inodes.

The magic command

df -i

indicated that I’d used 100% of inodes. So then my search to find out where I could get more inodes from began.

It turns out you can clean up some old files from git by issuing a command to do a garbage collection:

git gc

of course, you can’t do this without having *some* spare inodes. So I cleaned up some otherwise unused files from a few places. Running this command gave me 25 spare inodes. Hmmm… still looking…. Then it turned out that you need to ask gem to do some housekeeping. Gem keeps all the previous versions of RubyGems, and isn’t shy of using a lot of files. The magic words for gem are:

gem cleanup

which gave me over 20,000 free inodes after much work, ignoring a lot of errors and saying no to any extra removals that it questioned. Much better! Then I could complete the gem update I tried earlier. Sadly, there were still a lot of errors and it made it difficult to see what was going on, so I went about how to fix the invalid .gemspec format error. Fixing this required an update to gem itself, which can be run with

gem update --system

another command which may be useful is

gem install rubygems-update

running this solved the .gemspec format error, but now 4 of the gems complained about an invalid datestamp format inside the .gemspec. This also needed fixing, which can be done by executing

gem pristine

and then updating back to current versions, followed again by cleaning up the unwanted files. You can do all of them at once, e.g. ‘gem pristine oath2 faraday mini_magick uuid’ although I should mention that one didn’t work the first time for me so I had to do that step again. Then you can run ‘gem update’ followed by ‘gem cleanup’ and hey presto – totally clean install of gems!

Now it was back in business with running the system properly. I ran the db:migrate successfully this time, and when I ran the server it couldn’t connect to redis. Apparently I had killed the process and shouldn’t have. Easily fixed with the old:

nohup redis-server &

So now diaspora* is running on my server again! Sadly, I still can’t communicate with other pods, but at least I got back in to the swing of system admin again.

Diaspora* update

After some server tweaking to make better use of resources, and some code updates, my Diaspora* pod is much more stable again. Although it is written in Ruby, the footprint isn’t too large – and I am yet to resort to low memory versions of Ruby.

I have now posted from Diaspora* to both Twitter and Facebook. The interface looks more polished with every update, and new little features and bugfixes are coming along nicely. It is still very much in the Alpha stage of development, but is improving every day. The link to Facebook seems to be much more stable now. This may have been a temporary developer Facebook issue, as it would not previously allow me to enter the correct details for security.

I would also like to take a moment to remember Ilya Zhitomirskiy, co-founder and a developer of Diaspora*, who left us far too soon.

My Diaspora* Pod is up!

As a further foray in to open source software, and taking a more personal ownership to sharing data, I’ve had a look in to Diaspora*. Rather than dabbling round the edges, I’ve installed my own pod.

If you are unfamiliar with Diaspora*, what it is and what it stands for, check out Join Diaspora*.

So installing it is far from easy. It is still considered to be in “Alpha”, and so instructions are changing on a regular basis. Also, as it can integrate with other products such as Facebook and Twitter, instructions on how to do that has to keep pace with those products and their APIs.

Simplest and best advice for installation is follow the instructions on the GitHub wiki for Diaspora*, and it will get you a long way. There are plenty of sites telling you the instructions are wrong and provide alternatives, but this appears to be no longer required. The next thing is to take a good look at everything in the config directory, even if you don’t think you need to configure it. I had a silent failure (Diaspora* has a habit of just stopping when things go wrong and not always providing log file comments) due to a non-existent directory being required as a log file repository.

The biggest gap in instructions for me was how to configure Apache2 to do all the things you need it to. It largely came down to working out which modules need to be loaded to fix which not-so-helpful error messages.

I’ve managed to get Twitter to allow me to twet from Diaspora*, but I’m yet to have any luck with Facebook. It seems to revolve around a lack of callback URL, something which the Facebook interface is yet to let me do. This is one of the issues where if I let my side try to contact Facebook and it doesn’t work, Diaspora* aborts ungracefully, leaving me to have to clean up the still running processes. Failure to clean up the ruby processes causes great confusion on the next restart. I’m currently running the server on a detached screen which is a little bit manual, but it seems to work.