Make xerox b215 work with samba 4 again

Recently I bought xerox b215 (if you can, buy something other than xerox or hp) and wanted to make it scan to smb share. I already had configured samba in container using servercontainers/samba image.
So, it’s just to add another new share and configure user for scanner, right? Wrong!
It’ just didn’t worked. Thanks xerox’s engineers who decided not to burden end-user with diagnostic messages. It started scanning and after a second  returned back to the scan screen. Samba with log level 10 didn’t help me too, I just saw that client tried to connect and that all.
The tool which helped me is wireshark, I’ve found that after NTLMSSP_AUTH request from scanner samba sends STATUS_LOGON_FAILURE.

A little bit of “letsgoogleit” and voila ntlm auth = ntlmv1-permitted allowed me not to configure FTP for that lovely xerox.

Lenovo battery hack and whitelist at the same time

Recently I’ve got x230 laptop and have a plan to change buggy Intel Centrino 6205 adapter to something like Atheros, also I decided that it’s worth to have ability to use x220 like batteries, just in case.
To achieve that, I needed to flash patched firmware for EC controller (thinkpad-ec project) and modified bios (1vyrain project), but it was confusing, what should go first? Firstly I didn’t realised that thinkpad-ec flashes only EC firmware, it looked like EC mod will update bios to newer version than supported by 1vyrain, same time 1vyrain would update bios to version newer than supported by thinpkad-ec.
Finally, here is how to have EC mod together with patched BIOS on x230 laptop:
1. BIOS should be old enough to be compatible with  1vyrain and thinpkad-ec, at 2020-03-22 it should be not newer than 2.60 (1vyrain has requirements of more older bios than thinkpad-ec, requirements for 1vyrain patch can be found here)  otherwise it should be downgraded as described here.
2. Make bootable device with thinkpad-ec image, in BIOS set boot mode to ‘Legacy’ and update EC firmware.
3. Make bootable device with 1vyrain image, in BIOS set boot mode “UEFI only”, disable “Secure boot” and update BIOS.

In my case I ended with BIOS version 2.77 EC version 1.14.

How to just send logs from files to graylog2

That solution allows to read logs from file and just send them to remote syslog/graylog server. Logs will not influent on current syslog settings, you won’t need to filter them out of any syslog facility (like local7), all you need – the rsyslog (I’ve used v8).

My task was to send logs which wrote by java application (if I’m right log4j was used), they were rotated by logrotate with truncation, so few specific options were added.
I replaced %APP-NAME% in rsyslog’s template(RSYSLOG_SyslogProtocol23Format) to be able differentiate from which files log messages were read.

As for me, it’s better to write logs in format which allow them to be parsed easily or send them right to remote location , but if you need to do it quickly without modification of application it’s appropriate solution. Just copy config below in file like  /etc/rsyslog.d/99-graylog.conf and modify TARGET.ADDRESS, TARGET.PORT, app_ tag and File setting according to your environment.

module(load="imfile")

template(
name="SyslogProtocol23Format_modified" type="string"
string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%$.suffix% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n"
)

ruleset(name="sendToLogserver") {
action(type="omfwd" Target="TARGET.ADDRESS" Port="TARGET.PORT" Template="SyslogProtocol23Format_modified")
}

ruleset(name="app_logs") {
set $.suffix=re_extract($!metadata!filename, "(.*)/([^/]*)", 0, 2, "unknown.log");
call sendToLogserver
stop
}

input(
type="imfile"
File="/var/log/app_logs/*.log"
Tag="app_"
Ruleset="app_logs"
freshStartTail="on"
addMetadata="on"
)

In my case application wrote multi-line log messages, so startmsg.regex was used. Also logs were rotated by logrotate with truncate method, additional option reopenOnTruncate was used. So my input section looked like:

input(
type="imfile"
File="/var/log/app_logs/*.log"
Tag="app_"
Ruleset="app_logs"
freshStartTail="on"
addMetadata="on"
startmsg.regex="^[0-9]{4}-[0-9]{2}-[0-9]{2} "
reopenOnTruncate="on"
)

Fixing startup error of STMCubeMX on linux

After STMCubeMX was upgraded from version 4 to version 5, it can’t start. I’ve tried to reinstall it, but without result. Last messages in console after application stuck looks like:

2019-01-24 21:03:54,692 [INFO] PluginManage:339 - Loaded plugin projectmanager (category:projectmanager,tabindex:3)
2019-01-24 21:04:38,908 [ERROR] IntegrityCheckThread:90 - Cannot obtain updater plugin
2019-01-24 21:04:38,909 [INFO] IntegrityCheckThread:94 - End integrity checks thread
2019-01-24 21:04:38,909 [INFO] ThirdPartyDb:263 - Close Third Party DataBase File (/home/bob/.stm32cubemx/plugins/thirdparty/db/thirdparties_db.xml)

Same time java processes looks like:

bob 20652 102 1.5 5841340 127888 pts/3 Sl+ 21:03 2:41 java -jar STM32CubeMX
bob 20653 0.0 0.0 0 0 pts/3 Z+ 21:03 0:00 [STM32CubeMX] <defunct>

On the st forum I’ve found solution which had helped me, if you change tabindex parameter of com/st/microxplorer/plugins/tools/Plugin.properties in tools.jar to 6, STMCube will start to work.
Here is modified tools.jar

Fixing Gutenberg error “The editor has encountered an unexpected error”

After update to WP 5, I’ve faced with next issue, I’ve couldn’t add new post or edit existed. Looks like error happens because of misconfigured nginx and when new ‘Gutenberg’ editor is active (which is true by default for wordpress 5.0 and above).

Earlier I had nginx location / configured in next manner:

location / {
    try_files $uri $uri/ /index.php?$args;
}

Same configuration can be found on wordpress codex page:

And on nginx recipe page:

The issue caused by question sign in try_files directive, when $args is empty, index.php is called like this: “/index.php?”. Solution is simple:

$is_args
    “?” if a request line has arguments, or an empty string otherwise

After I changed location / block like this:

location / {
    try_files $uri $uri/ /index.php$is_args$args;
}

The problem is gone.

How to configure redmine service via terraform with persistent storage on amazon ECS

First of all, I have very little experience of AWS and terraform, so it can be obviously for them who have enough experience, but it definitely saved me a lot of time if I found article like that early.

It wasn’t simple to figure out how to run redmine container on ECS.
The main problem was – persistent storage. Redmine suppose that it have persistent disk storage which remain the same between service restarts. If you have your docker host it’s simply to map hipervisor’s directory inside of the container, but when your docker nodes can be added and removed dynamically you can lost data on disk which was generated by app.
Amazon provide few ways to have persistent storage such as S3, EBS or EFS.

By nature S3 is a storage which accessibly over http, so if your app haven’t integration with S3 API it can’t be used (except when you mount S3 via fuse fs for example).
EBS is a remote block storage, so you need to connect block device to docker host, mount it and map inside container before you will be able to use it.
EFS by nature is just a NFS.

I wanted to find solution which will be most natural as possible. I wanted to keep docker and redmine image untouched (ie avoid of additional plugins/scripts/packages installation). So, I decided not to use S3, because it need something like s3fs to make S3 storage available for redmine.
I decided not to use EBS, because I’ve found reports when EBS stuck attached to host and can’t be re-attached to another host until initial host reboots.
EFS looked perfect, it could be mounted from different hosts, it kept data during application/hypervisor life cycles. Moreover, even if I didn’t find a simple way to use EFS, only thing I needed was nfs-common package.

I was lucky, because at the Aug of 2018 amazon announced support of docker volumes and docker volumes plugins, docker itself can mount NFS inside containers since version 17.06 (I couldn’t found it in the change log, but if you google it, you will found a lot of references to that). So, it was exactly what I wanted, I faced only with one cons – lack of documentation. I needed to use terraform for redmine configuration and its documentation didn’t specify how to exactly pass driver_opts to docker volume configuration, so here is solution:

First you need to specify mount point in task-definition.json

"mountPoints":[
     {
       "sourceVolume": "redmine_storage",
       "containerPath": "/usr/src/redmine/files"
     }
 ]

And here is volume block from from terraform code for volume specification:

volume {
    name = "redmine_storage"
    docker_volume_configuration {
        scope         = "task"
        driver      = "local"
        driver_opts = {
            "type" = "nfs"
            "device" = "${var.efs_dns}:/"
            "o" = "addr=${var.efs_dns},nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2"
        }
 }
}

That’s all.
Code above is a part of redmine module, which have input variable efs_dns , so you can put your EFS address here if you configured it manually.

PS
Here you can find redmine S3 plugin, but I wanted to migrate existing redmine, so it looked like a lot of work, because I needed to modify rdemine’s DB and put files on S3 in manner which that plugin expects, so I decided that S3 not an option.

How to block IP ranges of specified autonomous system

If you want to prohibit access to your host for specified AS, you can use solution below. I made it some time ago, when I found out, that mail.ru hunting for hosts which help to bypass telegram censorship. It’s not perfect because I didn’t make much effort to it. Whois can return sub-networks and networks to which they belong in same response, so ipset set can contain duplicated ranges. Change ‘AS47764’ to AS which you want to block, ‘input_drop’ is an ipset set name.

ipset create input_drop hash:net comment
for i in $(whois -h whois.radb.net -- '-i origin AS47764' | grep 'route:'|cut -d : -f 2)
do
ipset add input_drop $i comment mail.ru
done
iptables -A INPUT -m set --match-set input_drop src -m comment --comment "DROP INPUT packets for AS47764" -j DROP

Also, i would recommend that solution, to make ipset rules persistent: https://github.com/BroHui/systemd-ipset-service

Galaxy S3: /efs/prox_cal doesn’t affect calibration settings under LineageOS

Few days ago I replaced front glass on samsung i9300 and flashed LineageOS 14.1. After that I’ve found that proximity sensor stays in triggered state, it may happened because of lack of experience (I’ve used too much UV-glue, so it was everywhere) or because of additional screen protector which been installed. Anyway, always-triggered-proximity-sensor made phone partially usable (you can’t cancel any call without pushing power button few times). I’ve found a lot of articles how to calibrate proximity sensor like this one. More over I’ve found that I shouldn’t do any calculation to update /efs/prox_cal, after auto-calibration /efs/prox_cal updated automatically (at least with kernel that shipped by default), but anyway it didn’t help me. Every reboot calibration  was reseted to zero.

For a first time, I’ve used proximity threshold value to fix proximity sensor, but later I saw that kernel driver read calibration directly from file and SELinux could be a reason why /efs/prox_cal haven’t effect.

Part that read calibration value looks like that:

#define CANCELATION_FILE_PATH "/efs/prox_cal"
...
int proximity_open_calibration(struct ssp_data *data)
{
 int iRet = 0;
 mm_segment_t old_fs;
 struct file *cancel_filp = NULL;
 
old_fs = get_fs();
 set_fs(KERNEL_DS);
 
cancel_filp = filp_open(CANCELATION_FILE_PATH, O_RDONLY, 0666);
 if (IS_ERR(cancel_filp)) {
 iRet = PTR_ERR(cancel_filp);
 if (iRet != -ENOENT)
 pr_err("[SSP]: %s - Can't open cancelation file\n",
 __func__);
 set_fs(old_fs);
 goto exit;
}

I’ve checked logcat and here is it:

05-06 21:29:12.916 3219 3219 W Binder:2377_A: type=1400 audit(0.0:39): avc: denied { read } for name="prox_cal" dev=mmcblk0p3 ino=46 scontext=u:r:system_server:s0 tcontext=u:object_r:efs_device_file:s0 tclass=file permissive=0

Definitely SELinux forbid reading of calibration file, I was surprised that SElinux capable to forbid kernel read call and now I feel a shame because usually I just disable it.

First I wanted to create new policy to allow reading of that file for kernel, but later I’ve found that /efs partition contains other calibration files, for example /efs/gyro_cal_data, I’ve checked security context of that files and found that it differs from /efs/prox_cal, it was u:object_r:sensors_data_file:s0 but prox_cal was created with default for /efs partition context u:object_r:efs_file:s0, so I’ve changed context:

# chcon u:object_r:sensors_data_file:s0 /efs/prox_cal

After that kernel started to load calibration value every boot. Looks like instructions like one mentioned above works for everyone who modified factory shipped prox_cal file with right security context, but I haven’t /efs/prox_cal before and it was created with wrong context.
I hope that story may help someone.

Dirty hack to add values mappings in Zabbix

“I’ll be brief.” ©
Here is two things about script published in ZBXNEXT-1424, first it can help you to automate creation of large mappings (and it’s cool), second it will broke your DB (not so cool, maaan).
When you will try to add mapping in broken DB you will see something like this:

poorzabbix

The “Error in query [INSERT INTO valuemaps (name,valuemapid) VALUES (‘Test mapping’,’50’)] [Duplicate entry ’50’ for key ‘PRIMARY’]” mean, that in table valuemaps you already have entry with valuemapid = 50. Why it happened i tell later after we fix DB.

To fix DB, you need to update few entries in table ‘idx‘, first update nextid where table_name = valuemaps:

mysql> update ids set nextid = (select max(valuemaps.valuemapid)+1 from valuemaps) where table_name = 'valuemaps';
Query OK, 1 row affected (0.22 sec)
Rows matched: 1 Changed: 1 Warnings: 0

Second update nextid for mappings:

mysql> update ids set nextid = (select max(mappings.mappingid)+1 from mappings) where table_name = 'mappings';
Query OK, 1 row affected (0.22 sec)
Rows matched: 1 Changed: 1 Warnings: 0

Here it is!

This happened because script does not update table idx. May be it’s ok for zabbix 2.0 that mentioned in feature request, but it’s broke database for zabbix 2.2 and newer. Unfortunately zabbix prior version 3.0 does not have API or ability to import mappings , so that script still useful.

Here is fixed script, i hope author will not offended at me:

#!/usr/bin/perl
 
use warnings;
use strict;
 
my $usage = "$0 valueMapName number newvalue [number2 newvalue2 [...]]
E.g.: 
 $0 'Alarm Status' 1 ok 2 unknown 3 stale 4 problem
 $0 'Aliveness' 0 dead 1 alive
";
 
my $valueMapName = shift() || die "No new valuemap name";
my @mapList = @ARGV;
die "No mappings given. Usage: $usage\n" if scalar(@mapList) == 0;
 
 
my $isEvenNumber = scalar(@mapList) % 2 == 0;
die "Must give mapping->value pairs. Usage: $usage\n" if not $isEvenNumber;
my %mappings = @mapList;
 
my $newValueMapId = int(qx/mysql -N -s -e 'select nextid from zabbix.ids where field_name = "valuemapid"'/) ||
die("Can't fetch max valuemapid\nUsage: $usage\n");
$newValueMapId++;
my $newMappingId = int(qx/mysql -N -s -e 'select nextid from zabbix.ids where field_name = "mappingid"'/) ||
die("Can't fetch max mappingid\nUsage: $usage\n");
$newMappingId++;
 
eval {
 my $valueMapCmd = qq/mysql -e "insert into zabbix.valuemaps (valuemapid, name) values ('$newValueMapId', '$valueMapName');"/;
 print "$valueMapCmd\n";
 system $valueMapCmd;
 eval {
 for my $from (keys %mappings) {
 my $to = $mappings{$from};
 my $mappingCmd= qq/mysql -e "insert into zabbix.mappings (mappingid, valuemapid, value, newvalue) values ('$newMappingId', '$newValueMapId', '$from', '$to');"/;
 print "$mappingCmd\n";
 system $mappingCmd;
 $newMappingId++;
 }
 };
 if ($@) {
 die "something went wrong inserting into mappings $@";
 }
};
if ($@) {
 die "something went wrong inserting into valuemaps $@";
}
 
my $valueMapUpdCmd = qq/mysql -e 'update zabbix.ids set nextid = "$newValueMapId" where field_name = "valuemapid";'/;
print "$valueMapUpdCmd\n";
system $valueMapUpdCmd;
$newMappingId--;
my $mappingUpdCmd = qq/mysql -e 'update zabbix.ids set nextid = "$newMappingId" where field_name = "mappingid";'/;
print "$mappingUpdCmd\n";
system $mappingUpdCmd;

 

LVM recovery

Few days ago i made mistake and forced fsck to check partition that contain LVM instead of logic volume, as result i got broken LVM metadata. I was unable to see volume group an logic volumes.
pvs output looked like that:

# pvs -v

Scanning for physical volume names
Incorrect metadata area header checksum

I tried to run pvck but it did not help me, it founded corrupted metadata but did not repair LVM:

# pvck -d -v /dev/md5
Scanning /dev/md5
Incorrect metadata area header checksum
Found label on /dev/md5, sector 1, type=LVM2 
Found text metadata area: offset=4096, size=193024
Incorrect metadata area header checksum

Finally i founded that it’s possible to make backups of LVM metadata and restore it when needed, but i think that i had only broken LVM with broken metadata.
It’s hard to describe how happy I was when I found that by default LVM create backups of metadata when you make any changes. I found it into /etc/lvm/backup dir, after that recovery become easy task, first i recreate physical volume:

pvcreate -u b3Lk2a-pydG-Vhf3-DSEJ-9b84-RLm9-UEr6r3 --restorefile /etc/lvm/backup/vg-320 /dev/md5

UUID can be founded in pv section into metadata file:

 physical_volumes {
 
 pv0 {
 id = "<strong>b3Lk2a-pydG-Vhf3-DSEJ-9b84-RLm9-UEr6r3</strong>"
 device = "/dev/md5" # Hint only

Next i restored volume group:

vgcfgrestore -f /etc/lvm/backup/vg-320 vg-320

After that logical volumes became visible:

# lvs
 LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
 root vg-320 -wi-a--- 15.00g 
 swap vg-320 -wi-a--- 1.00g 
 var vg-320 -wi-ao-- 200.00g 
 zoneminder vg-320 -wi-a--- 15.00g

After reinitialization with vgscan -v && vgchange -ay commands, volume groups ready for fsck.