macOS: SystemUIServer eating your CPU? Check your Ethernet connection!

Recently, the SystemUIServer process on my MBP running macOS Sierra has started “eating” a lot of CPU, slowing down the whole machine, even making the clock in the top menu bar stop working properly. It usually started using one-digit percentages of the available CPU power, then growing to 10%, 15%, 20%, up to well above 60%, sometimes even 80% and more! It wasn’t a steady growth – it sometimes shrank again, just to grow even further.

The only apparent remedy was to kill the SystemUIServer process (e.g. using the Activity Monitor) from time to time (i.e. every 30 minutes -> there are also scripts to automatically restart SystemUIServer). Its CPU usage then reset to a low one-digit number.

Taking a closer look at the process in the Activity Monitor, I then noticed that the number of (used) ports (so-called “Mach ports“) by the process were steadily growing, once SystemUIServer was started. This was weird, pointing to some kind of leakage. Typically, for a CPU load of around 50%, more than 5000 Mach ports were used.

By coincidence, I then noticed that, unlike expected, my MBP wasn’t actually using Ethernet, but only WiFi. Further investigation then hinted that the according Gigabit Ethernet port on my HP 1810 switch was apparently malfunctioning (or even dead): In the macOS Network Preferences, the Thunderbolt-Ethernet connection was constantly shown as red/disconnected, although the OS was apparently trying to establish a connection again and again (and failed). First, I even suspected a problem with the Thunderbolt-Gigabit-Ethernet adapter itself (it wasn’t the problem here, the adapter seemed to work fine with another Mac and connection).

The solution to this problem thus was:

  • Connect the Ethernet cable to another, working Ethernet port: Now the SystemUIServer process consumes less than 0.1% CPU again and roughly 400 ports only, both with and without additional WiFi.
    Note that both the problem and this solution are reproducible.

Lessons learned:

  • Sometimes, very unexpected, seemingly unrelated and “small” problems can have big (negative) effects.
  • Sometimes you need a bit of luck to find the cause of a problem (a web search didn’t bring up the above hint, rather suggested updating or removing faulty apps, buggy extensions and menu widgets. I thus already tried removing or updating some of the suspected apps, extensions and widgets.)
  • Ports of HP 1810 switches can actually break/fail! Remember the saying: “I got 99 problems, but a switch ain’t one!” – well, in this case, the faulty switch was actually part of the problem and even the initial trigger of the problem! Also remember that HP offers a lifelong warranty on its (good ol’) 1810 switches.
  • Extra points for you, further research: The fact that the SystemUIServer allocates more and more Mach ports if there’s a malfunctioning Ethernet port (i.e. faulty Ethernet connection or faulty handling of a faulty Ethernet connection by the Thunderbolt-Gigabit-Ethernet adapter) is hinting that this might be an attack vector for a (new?) DoS attack. Perhaps not an easily exploitable one (on the Ethernet or MAC layer, even), but it’s nonetheless something that should actually be handled gracefully by SystemUIServer, not leading to more and more CPU and system resources being consumed.
    If you have time to research this further, let me know about your findings!

The ‘All-in-One WP Migration’ plugin is all you need to migrate your WordPress blog

This blog has just been migrated to a newer and (much) faster host node running Proxmox 5 with ZFS.

Therefore, I was looking for the best method to migrate a WordPress blog to another server (and/or database and/or directory and/or URL).

Do you remember the times when migrating a WordPress blog was rather tedious, involving many manual steps, despite (other) handy tools like WP-CLI? Apparently, this is no longer needed, as all you need is the following plugin:

All-in-One WP Migration

This plugin’s export and import functionality takes care of all the required configuration and path adjustments, allowing you to easily migrate a blog with up to 512 MB data. The steps are thus:

  1. Install the All-in-One WP Migration plugin on your current WordPress site
  2. Use the plugin to export all your data, plugins, themes, configuration etc., e.g. as a downloadable file.
    Note: In particularly tricky cases you can also manually replace certain strings in the db or exclude specific data and files.
  3. Setup a new vanilla WordPress installation at another location (server, directory). You’ll need a database and the WP installation files for this.
  4. Install the All-in-One WP Migration plugin on your new WordPress site
  5. Use the plugin on your new site to import the previously exported data from your old site

It’s hard to believe, but that’s really it!

How to upgrade Zimbra/ZCS 8.8 GA from Ubuntu 14.04 LTS to 16.04 LTS

When upgrading Ubuntu 14.04 LTS to 16.04 LTS the usual way, ‘do-release-upgrade’ will by default remove 3rd party packages. For a Zimbra/ZCS server this means that Zimbra/ZCS packages would be automatically removed when upgrading to Ubuntu 16.04 LTS.

Luckily, there’s a neat little trick to prevent ‘do-release-upgrade’ from removing Zimbra/ZCS packages during the upgrade. Here’s how to do it:

  1. Open two screen sessions (or two terminal sessions) on your Zimbra/ZCS server
     $ screen -R
  2. Block at least the SMTP and SMTPS ports (to prevent clients from accessing the server, consider blocking the IMAP and IMAPS ports too, or all ports but SSH and port 1022) for your Zimbra/ZCS server on your firewall (to stop the delivery of messages).
  3. Backup the server or make a snapshot, just in case anything goes wrong
  4. Stop Zimbra/ZCS:
     $ sudo /etc/init.d/zimbra stop
  5. Make sure the system is current:
     $ sudo apt-get update && sudo apt-get upgrade
  6. Start the Ubuntu distro upgrade process:
     $ sudo do-release-upgrade

    (if you don’t have do-release-upgrade, you have to execute ‘sudo apt-get install update-manager-core’ first)

  7. When prompted by ‘do-release-upgrade’ that the third party sources have been disabled, re-enable those third party sources. To do this, open ‘/etc/apt/sources.list.d/zimbra.list’ with an editor (e.g. nano, vim) in another terminal/screen session and change its content from:
    # deb [arch=amd64] https://repo.zimbra.com/apt/87 xenial zimbra # disabled on upgrade to xenial
    # deb-src [arch=amd64] https://repo.zimbra.com/apt/87 xenial zimbra # disabled on upgrade to xenial

    to

    deb [arch=amd64] https://repo.zimbra.com/apt/87 xenial zimbra
    deb-src [arch=amd64] https://repo.zimbra.com/apt/87 xenial zimbra

    Then return to the first terminal/screen session and let ‘do-release-upgrade’ continue with the upgrade process (i.e. hit ‘enter’).

  8. When you get informed about the packages that will be removed (BTW, in the detail view you can see that the Zimbra packages will now be upgraded, not removed) and upgraded and asked whether you want to start the upgrade, confirm this.
    Note: Don’t worry about minor error messages like:E: changelog for this version is not (yet) available; try https://launchpad.net/ubuntu/+source/zimbra-perl-socket/+changelogYou can safely ignore them.
  9. Agree to all suggestions by ‘do-release-upgrade’ (e.g. the removal of files in /var/log/sysstat/ and whether you allow ssh to be restarted). In my case, it was also fine to go with the maintainer versions of the config files.
  10. When the upgrade process is finished, let ‘do-release-upgrade’ reboot the system.

After this, Zimbra/ZCS should work nicely again, on Ubuntu 16.04 LTS.

Note: It can take Zimbra/ZCS quite some time to properly start all its services (it’s Java, after all). Sometimes, the output of ‘$ sudo /etc/init.d/zimbra status’ and what’s listed in the service monitoring section of the Zimbra/ZCS admin webUI can thus be inconsistent. Sometimes, it’s even necessary to stop and start Zimbra/ZCS a couple of times (with ‘/etc/init.d/zimbra’) after an upgrade until all services run nicely.
Also note that Zimbra’s new dedicated ‘imapd’ service won’t run properly, unless it’s configured manually, as shown in the Zimbra Collaboration Administrator Guide version 8.8.3. If it doesn’t run properly, this service will simply be ignored in a single server setup, your Zimbra/ZCS will thus likely work as usual.

If there are any problems or if you want to be extra cautious, you can also additionally download Zimbra/ZCS 8.8 for 16.04 LTS manually and run its installer again:

$ sudo ./install.sh

If everything is fine, unblock the SMTP and SMTPS (and IMAP and IMAPS) ports again.

If things aren’t fine, simply roll-back from the snapshot or restore the whole server from the backup.

You might then want to try a fresh install according to the official Zimbra/ZCS migration manual: How to move ZCS to another server.

(Source: Kudos to vchong68 for his valuable hint in his forum post)

Solution for: Proxmox backup error due to iothread=1

If you see the following error when trying to backup a KVM VM image on Proxmox:

ERROR: Backup of VM 100 failed – disk ‘scsi0’ ‘zfsvols:vm-100-disk-1’ (iothread=on) can’t use backup feature currently. Please set backup=no for this drive at /usr/share/perl5/PVE/VZDump/QemuServer.pm line 77. INFO: Backup job finished with errors TASK ERROR: job errors

edit /etc/pve/qemu-server/100.conf, look for a line similar to

scsi0: zfsvols:vm-100-disk-1,iothread=1,size=70G

and change it to

scsi0: zfsvols:vm-100-disk-1,iothread=0,size=70G

Afterwards, it’s possible to backup the VM.

Using multiple Skype accounts on macOS

With the recent changes in Skype, the “traditional” method of running several Skype instances using different system users doesn’t work anymore (as the the new authentication dialog strangely doesn’t get the focus anymore).

So, in order to use several Skype accounts on macOS (formerly known as Mac OS X), do the following:

  1. Open the Script Editor (in the ‘Utilities’ folder in the ‘Applications’ folder)
  2. Create a new script with the following content:
    do shell script "open -na /Applications/Skype.app/Contents/MacOS/Skype --args -DataPath '/Users/your_system_user/Library/Application Support/Skype_any_identifier'"

    Replace your_system_user by your regular macOS user account (see ‘whoami’ in Terminal)
    Replace _any_identifier by the according Skype account name, e.g. _myskypename (doesn’t really matter what, just don’t use an empty string)
    Please mind the double quotes and single quotes (important!)

  3. Save the script as an application: File.. Save.., choose “Application” as file format, give it a name and store it e.g. in your home directory or in the Applications folder

Repeat these steps for any of your Skype accounts, giving each Skype account a different Skype_any_identifier. You can then start the according Skype instances by double clicking on the according app.

Explanation:

The above script starts a new instance of Skype (which would otherwise be prevented), using the -n argument. Each instance of Skype gets its own directory to store the according account data, using the -DataPath argument.

How to check filesystems in a qcow2 image

A useful post how to fsck (check and fix) a filesystem in a qcow2 image (as typically used for KVM VMs, e.g. in Proxmox):

How to recover a qcow2 file using fsck

On Proxmox or Debian, one does the following:

Attention:

  • Make sure the according VM isn’t running, i.e. the partition not mounted
  • Adjust the commands below to match your system, use the correct qcow2 image, use the correct fsck-variant, fsck the correct filesystem, note that -p tries to automatically fix errors!
# modprobe nbd max_part=8
# qemu-nbd --connect=/dev/nbd0 /var/lib/vz/images/100/vm-100-disk-1.qcow2
# fdisk -l /dev/ndb0
/dev/nbd0p1            2048     7813119     3905536   82  Linux swap / Solaris
/dev/nbd0p2   *     7813120   119537663    55862272   83  Linux
# fsck.ext4 /dev/nbd0p2
# fsck.ext4 -p /dev/nbd0p2
# qemu-nbd --disconnect /dev/nbd0

Like this, one doesn’t need to boot the VM using a boot ISO/CDROM and can fix the filesystem right from the host node.

 

Combining the Eisenhower Matrix and Kanban on Trello

I like the Eisenhower Matrix a lot: In everyday life, the simple concepts often tend to be the most practical ones, and in the end, usually also the most useful and powerful ones. Remember: “Everything should be made as simple as possible, but not simpler” (although that’s also a reminder to not over-simplify things).

Here’s an idea how to combine the concept of 2-dimensional triage (Eisenhower Matrix) with the idea of JIT workflows (JIT processes from supply chain to production to delivery) in Kanban / Kanban in software development, using Trello:

eisenhower_matrix_combined_with_kanban_on_trello

The top Kanban workflow would thus be:

1. INBOX: If urgent & important: DO!

2. If urgent & not important: DELEGATE!

3. If not urgent & important: DECIDE+PLAN!

4. Rest: Not urgent & not important: ELIMINATE!

(& := logical AND)

This would guarantee, that first, all incoming tasks/requests (e-mails, phone calls, mail, visitors, time-triggered events, other events, ..) would get proper (highest) attention and get collected in the INBOX list. They would then be triaged (qualified) and, if appropriate, distributed/moved to the other lists accordingly. If nothing else matches, they would end up in the low priority box (not urgent AND not important), i.e. the equivalent of a “trash”.

In this order, it’s a failsafe workflow, where all incoming tasks are treated as highest priority tasks at first by default. In some seldomn special cases, usually just temporarily, one could also imagine that reversing the workflow would make more sense (e.g. in a situation of a foreseeable flood of low-prio incoming requests, when you only have time to pick the high-priority ones out of these and leave almost all tasks in the “trash” by default -> this is not failsafe, of course!)

Integrating a particular product’s Kanban workflow would require adding further “sub-lists” for each of the steps of the product lifecycle. Or one could also think of using a separate board for each of the 4 states of the Eisenhower matrix, with lists representing the Kanban states/stages connected through Kanban JIT processes.

Note that each team member can (and should) have her/his own Eisenhower sub-matrix in addition, also combined with her/his own Kanban sub-workflow.


Side note: One could also extend the above board and make it more fine-grained by subdividing each list into the following 5 GTD (Getting Things Done) sub-workflows:

capture, clarify, organize, reflect, and engage

These would ideally be represented by sub-lists, which don’t exist in Trello (yet?). Currently, one would thus have to improvise a bit and e.g. use cards for them, using comments as “sub-cards” – or add further lists, e.g. “1.1 capture”, “1.2 clarify”, “1.3 organize”, “1.4 reflect”, “1.5. engage”.

One could also treat the above combination of the Eisenhower Matrix and Kanban as an application of the 5 GTD workflows and do without making those explicit. At the moment, I favor this view and, for simplicity, don’t recommend visualizing these GTD “meta-workflows”.


 

Disclaimer: The ideas proposed here are “work in progress”, likewise this post. This is just one example of many imaginable ones. I’ll probably edit this post or clarify, correct or extend it in follow-up posts.

#32c3 presentations to watch (note to self)

Overview of recorded presentations:

https://media.ccc.de/b/congress/2015

My list of particularly interesting presentations (mostly for myself – disclaimer: I haven’t watched all of these presentations yet as I didn’t make it to Hamburg this year):