advogato

Posts to be exported to the free software community at Advogato.

check-all-the-things (aka cats, Meow!) is a tool that aims to make it easy to know which tools can be used to check a directory tree and to make it easy to run those tools on the directory tree. The tree could either be a source tree or a build tree or both. It aims to check as much of the tree as possible so the output can be very verbose and have many false positives. It is not for the busy, lazy or noise intolerant. It runs the checks by matching file names and MIME types against those registered for a list of checks. Each check has a set of dependencies, flags, filename wildcards, MIME type wildcards, comments and prerequisite commands. By default it:

  • doesn't check file MIME types as this is slower
  • shows which command is currently running
  • limits check output to 10 lines
  • hides checks that output nothing
  • kills checks when interrupted with Ctrl+C
  • exits when interrupted twice in quick succession
  • outputs various remarks at the end

It runs all checks for the current distro/release except:

  • dangerous ones that execute code in the current dir
  • ones that modify files in the current dir
  • ones that access the network (if there is no default route)
  • ones that need work to be usable
  • ones that need a human to run them

There are command-line options to customise the behaviour and automatic bash shell completion via argcomplete. There are 177 checks (including TODO ones) in 73 different categories. There are an additional 224 not-well-specified TODO items for new checks in comments. It is exceptionally easy to add new checks once one knows how to use the tool one wants to add.

At this point in time it is probably not a good idea to run it in an untrusted directory tree for several reasons:

  • there could be unknown vulnerabilities in the tools used
  • there could be unknown interactions with interpreters (known ones worked around)
  • there could be some commands doing unknown code execution
  • there could be other weirdness in some layers
  • there is no automatic sandboxing at all yet

The project initially started as really hacky wiki page full of commands to run. At some point I figured it was time to make this actually be maintainable and started on a project to do that. At around the same time Jakub Wilk was working on maquack to replace the wiki page. Somehow I found out about it and talked to him about it. It was vastly less hacky than my version so I ended up taking it over and continuing it under the check-all-the-things name. I polished it for the last two years and finally released it into Debian unstable during DebCamp16.

Posted Mon Jul 4 18:10:55 2016 Tags: advogato

Today was day one of the DebConf16 Open Festival and I attended the open hardware panel, part of the talk about Code For South Africa, shirish's experiences and the DebConf new folks session.

The open hardware panel was a wide ranging discussion between bdale, Andy and indiebio. bdate talked about the experiences he has had with his rocketry hardware. bdale said "Make concious decisions about what you are buying", referencing a case where he investigated, found a GPL violation and didn't buy. Various people care about openness of different layers of the hardware. Off-the-shelf products are very strongly integrated, which is great for makers but means that people who care about lower layers like CPU micro-architecture aren't able to participate. Andy said "We are just beginning to come out of the shareware stage [of open hardware]". bdale mentioned the companies who do hardware production as a service from design files. Later in the pub some folks mentioned j-core, an open re-implementation of SuperH processors.

I missed most of the code4sa talk unfortunately, but it was about government services and open data.

shirish covered his journey through life to Debian. His youth, how satellite TV and knowledge of the outside world came to India around the time of the Iraq war. His experience accessing the Internet for the first time, uncensored vs the usual censorship in India's media. His experiences of Windows 95 viruses and crashes. He learned of PCTwist Linux through a magazine cover. His initial install was not a success but eventually managed to break through and install a desktop, but experienced network and other issues. Eventually he encountered Ubuntu and began contributing bug reports. His experiences there led him to Debian. He began blogging about Debian. In the last few years he and others have been going around the country doing mini-DebConfs at institutes around India. The first question was predictably about having a DebConf in India and how shirish might like to get more involved. DebConf in India sounds like a possibility some day and shirish was thinking about getting involved in publicity, marketting and the Debian installer.

The DebConf new folks session was a great intro to DebConf for folks new to the community. There were some quite excellent touches added to this year's version of the event by indiebio and Rhonda.

I also got some things done. Usual spam reporting. Reviewed wiki RecentChanges. Talked to the chromium-bsu/MacPorts maintainer about AX_CHECK_GL brokenness. Filed Debian wishlist bug #829292 asking to update autoconf-archive. Redirected a Hurd porterbox request to the exodar admin and quickly found out I was wrong to do that, rectified. Then we found out the LDAP sync to exodar was broken. Replied to someone who intends to sell Debian pre-installs. Thanked BunsenLabs folks for joining the derivatives list. Applied reproducible builds patch for cats from Chris Lamb. Heard about awesome new terminal-mode screensaver. Moo! Prepared a blog post about check-all-the-things.

Posted Sun Jul 3 05:17:51 2016 Tags: advogato

Apply wget security update to mentors.debian.net. Poke the DebConf video team about archiving the one Debian & stuff podcast episode. Discuss exclusion, privilege & DebConf. Usual spam reporting. Review wiki RecentChanges. Fix some typos from RecentChanges. File Debian wishlist bug #829177 against bugs.debian.org. Mention the recent post about breaking Android full-disk encryption on the exploits Debian wiki page. Answer questions about recommended build configuration for chromium-bsu from the maintainer of it in FreeBSD. Mention that HTML SRI could help secure initial Debian downloads. File Debian bug #829199 against file and add workaround in derivatives census. Report broken boss-gnome source package to BOSSLinux folks on IRC. Delete and re-download Parsix apt folder to stop hash sum mismatches. File Debian minor/wishlist bugs #829209/#829211/#829212 against cypher-lint. Add a porterbox guest account for one of the RTC GSoC students. Extend the expiry of another guest account. Direct query about the excuses HTML to the release team. File Debian bug #829241 against fonts-play. Discuss accessibility, life, the universe and rakia.

Posted Sat Jul 2 05:21:40 2016 Tags: advogato

Usual spam reporting. Review wiki RecentChanges. Provide feedback for the staging site of the new codebase for screenshots.d.n. Redirect bugs-search.d.o complaint to the BTS maintainers. Point out pastebinit already supports fpaste.org. Polish chromium-bsu, make a new upstream release to fix Debian RC bug #822711. Upload screenshot of chromium-bsu menu. Notify chromium-bsu package maintainers in other distros (hug whohas). Avoid checking WAV files for spelling errors in cats. Make the old PTS download i18n data over https. File #829092 to get the per-package i18n data to use https for links. Point someone on mentors to the Debian PHP group wiki page.

Posted Fri Jul 1 05:38:56 2016 Tags: advogato

Redirect one person contacting the Debian sysadmin and web teams to Debian user support. Review wiki RecentChanges. Usual spam reporting. Check and fix a derivatives census issue. Suggest sending the titanpad maintainence issue to a wider audience. Update check-all-the-things and copyright review tools wiki page for licensecheck/devscripts split. Ask if debian-debug could be added to mirror.dc16.debconf.org. Discuss more about the devscripts/licensecheck split. Yesterday I grrred at Debian perl bug #588017 that causes vulnerabilities in check-all-the-things, tried to figure out the scope of the issue and workaround all of the issues I could find. (Perls are shiny and Check All The thingS can be abbreviated as cats) Today I confirmed with the reporter (Jakub Wilk) that the patch mitigates this. Release check-all-the-things to Debian unstable (finally!!). Discuss with the borg about syncing cats to Ubuntu. Notice autoconf/automake being installed as indirect cats build-deps (via debhelper/dh-autoreconf) and poke relevant folks about this. Answer question about alioth vs debian.org LDAP.

Posted Wed Jun 29 20:49:00 2016 Tags: advogato

Beat head against shiny cats (no animals were harmed). Discuss the spice of sillyness. Forward a wiki bounce to the person. Mention my gobby git mail cron job. Start adopting the adequate package. Discuss cats vs licensecheck with Jonas. Usual spam reporting. Review wiki RecentChanges. Whitelisted one user in the wiki anti-spam system. Finding myself longing for a web technology. Shudder and look at the twinklies.

Posted Tue Jun 28 19:14:00 2016 Tags: advogato

Usual spam reporting. Review wiki RecentChanges. Rain glorious rain! Err... Update a couple of links on the debtags team page. Report Debian bug #828718 against tracker.debian.org. Update links to debtags on DDPO and the old PTS. Report minor Debian bug #828722 against debtags.debian.org. Update the debtags for check-all-the-things. More code and check fixes for check-all-the-things. Gravitate towards the fireplace and beat face against annoying access point, learn of wpa_cli blacklist & wpa_cli bssid from owner of devilish laptop. Ask stakeholders for feedback/commits before the impending release of check-all-the-things to Debian unstable. Meet developers of the One^WGNU Ring, discuss C++ library foo. Contribute some links to an open hardware thread. Point out the location of the Debian QA SVN repository. Clear skies at night, twinkling delight.

Posted Mon Jun 27 20:25:04 2016 Tags: advogato

Review, approve chromium, gnome-terminal and radeontop screenshots. Disgusted to see the level of creativity GPL violators have. Words of encouragement on #debian-mentors. Pleased to see Tails reproducible builds funding by Mozilla. Point out build dates in versions leads to non-reproducible builds. Point out apt-file search to someone looking for a binary of kill. Review wiki RecentChanges. Alarmingly windy. Report important Debian bug #828215 against unattended-upgrades. Clean up some code in check-all-the-things and work on fixing Debian bug #826089. Wind glorious wind! Much clearer day, nice view of the mountain. More check-all-the-things code clean up and finish up fixing Debian bug #826089. Twinkling city lights and more wind. Final code polish during dinner/discussion. Wandering in the wind amongst the twinklies. Whitelisted one user in the wiki anti-spam system. Usual spam reporting.

Posted Sun Jun 26 19:31:25 2016 Tags: advogato

Review wiki RecentChanges since my bookmark. Usual spam reporting. Mention microG on #debian-mobile. Answer pkg-config question on #debian-mentors. Suggest using UUIDs in response to a debian-arm query. Reported Debian bug #828103 against needrestart. A giant yellow SOS crane between the balcony hacklab and a truly misty city. Locate the 2014 Debian & stuff podcast on archive.org. Poke the SPARC porters in response to a suggestion on debian-www. Mention systemctl daemon-reload wrt buildd service changes. Automate updating some extension lists from check-all-the-things. Reported wishlist Debian bug #828128 against debsources. Engage lizard mode! Wish for better display technology. Nice vegetarian food with nice folks and interesting discussions with interesting locals. Polish and release check-all-the-things. Close bugs I forgot to close in the changelog. Add link to debian-boot on Debootstrap wiki page. Notice first mockup of a theme for Debian stretch. Answer a question about package naming on #debian-mentors. Discuss the future of cross compilation on Debian. Notice a talk about FOSSology & update a wiki page. Mention AsteroidOS and MaruOS on the mobile wiki page. Contemplate how close to the FSDG Debian might be and approaches to improving that.

Posted Sat Jun 25 19:49:32 2016 Tags: advogato

Hating jetlag based headache. Disturbed to see the Brexit result. Review wiki RecentChanges. Answer some questions about Launchpad on #debian-mentors. Whitelisted one user in the wiki anti-spam system. Reviewed and sponsored yamllint 1.2.2-1 upload. Noted OFSET repo is broken and updated Freeduc info. Noted the Epidemic-Linux website is having database issues. Noted that Facebook finally completely dropped their RSS feeds, dropped Facebook RSS feed URL generation from the Debian derivatives census scripts and notified the affected derivatives. Cleared up Tanglu hash sum mismatches again. Minor changes to Planet Debian derivatives. Enjoyed the photos from Valessio. Hazy city away from the mountain and tablecloth clouds flowing over the mountain on the way to a pub lunch. Jet lag headaches seem to be subsiding thankfully. Ping someone generating a bounce when changing their SSH key. Mention autorevision and other suggestions in an IRC discussion about mesa & reproducible builds. Review some DebConf16 announcements and add minor fix. Push out some TODO items to check-all-the-things. Ask for a dd-list for the GCC 6 transition. Usual spam reporting throughout the day via manual List-Archive copy-paste, feeding mboxen to my report-spam-debian-lists and report-spam-debian-bugs scripts and manual BTS clicks. Usual wondering why there isn't an RFC for MUA spam reporting. Disturbed by the sudden appearance of an astronautess in the orga room but placated by a plentiful supply of crisps. Ask x32 folks about debian-x32.org vs x32 on ports.d.o. Glad to just avoid the room shuffle dance. Finish mime support for check-all-the-things. Disappointed that piz.za does not actually resolve. Amused by pollito's virtual tour of UTC. Completely stuffed full of Butleritos.

Posted Fri Jun 24 17:46:11 2016 Tags: advogato