Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) // SPDX-License-Identifier: GPL-2.0-only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) /*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3)  * Copyright (C) 2016,2017 ARM Limited, All Rights Reserved.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4)  * Author: Marc Zyngier <marc.zyngier@arm.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5)  */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) #include <linux/interrupt.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) #include <linux/irq.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) #include <linux/irqdomain.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) #include <linux/msi.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) #include <linux/sched.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) #include <linux/irqchip/arm-gic-v4.h>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) /*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16)  * WARNING: The blurb below assumes that you understand the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17)  * intricacies of GICv3, GICv4, and how a guest's view of a GICv3 gets
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18)  * translated into GICv4 commands. So it effectively targets at most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19)  * two individuals. You know who you are.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21)  * The core GICv4 code is designed to *avoid* exposing too much of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22)  * core GIC code (that would in turn leak into the hypervisor code),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23)  * and instead provide a hypervisor agnostic interface to the HW (of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24)  * course, the astute reader will quickly realize that hypervisor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25)  * agnostic actually means KVM-specific - what were you thinking?).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27)  * In order to achieve a modicum of isolation, we try to hide most of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28)  * the GICv4 "stuff" behind normal irqchip operations:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30)  * - Any guest-visible VLPI is backed by a Linux interrupt (and a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31)  *   physical LPI which gets unmapped when the guest maps the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32)  *   VLPI). This allows the same DevID/EventID pair to be either
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33)  *   mapped to the LPI (host) or the VLPI (guest). Note that this is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34)  *   exclusive, and you cannot have both.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36)  * - Enabling/disabling a VLPI is done by issuing mask/unmask calls.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38)  * - Guest INT/CLEAR commands are implemented through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39)  *   irq_set_irqchip_state().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41)  * - The *bizarre* stuff (mapping/unmapping an interrupt to a VLPI, or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42)  *   issuing an INV after changing a priority) gets shoved into the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43)  *   irq_set_vcpu_affinity() method. While this is quite horrible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44)  *   (let's face it, this is the irqchip version of an ioctl), it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45)  *   confines the crap to a single location. And map/unmap really is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46)  *   about setting the affinity of a VLPI to a vcpu, so only INV is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47)  *   majorly out of place. So there.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49)  * A number of commands are simply not provided by this interface, as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50)  * they do not make direct sense. For example, MAPD is purely local to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51)  * the virtual ITS (because it references a virtual device, and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52)  * physical ITS is still very much in charge of the physical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53)  * device). Same goes for things like MAPC (the physical ITS deals
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54)  * with the actual vPE affinity, and not the braindead concept of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55)  * collection). SYNC is not provided either, as each and every command
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56)  * is followed by a VSYNC. This could be relaxed in the future, should
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)  * this be seen as a bottleneck (yes, this means *never*).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59)  * But handling VLPIs is only one side of the job of the GICv4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60)  * code. The other (darker) side is to take care of the doorbell
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)  * interrupts which are delivered when a VLPI targeting a non-running
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62)  * vcpu is being made pending.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64)  * The choice made here is that each vcpu (VPE in old northern GICv4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65)  * dialect) gets a single doorbell LPI, no matter how many interrupts
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66)  * are targeting it. This has a nice property, which is that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67)  * interrupt becomes a handle for the VPE, and that the hypervisor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68)  * code can manipulate it through the normal interrupt API:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70)  * - VMs (or rather the VM abstraction that matters to the GIC)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71)  *   contain an irq domain where each interrupt maps to a VPE. In
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)  *   turn, this domain sits on top of the normal LPI allocator, and a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73)  *   specially crafted irq_chip implementation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75)  * - mask/unmask do what is expected on the doorbell interrupt.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77)  * - irq_set_affinity is used to move a VPE from one redistributor to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78)  *   another.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79)  *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80)  * - irq_set_vcpu_affinity once again gets hijacked for the purpose of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81)  *   creating a new sub-API, namely scheduling/descheduling a VPE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82)  *   (which involves programming GICR_V{PROP,PEND}BASER) and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83)  *   performing INVALL operations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84)  */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) static struct irq_domain *gic_domain;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) static const struct irq_domain_ops *vpe_domain_ops;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88) static const struct irq_domain_ops *sgi_domain_ops;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) static bool has_v4_1(void)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) 	return !!sgi_domain_ops;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) static int its_alloc_vcpu_sgis(struct its_vpe *vpe, int idx)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) 	char *name;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) 	int sgi_base;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 	if (!has_v4_1())
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) 		return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 	name = kasprintf(GFP_KERNEL, "GICv4-sgi-%d", task_pid_nr(current));
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) 	if (!name)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 	vpe->fwnode = irq_domain_alloc_named_id_fwnode(name, idx);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) 	if (!vpe->fwnode)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) 	kfree(name);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) 	name = NULL;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) 	vpe->sgi_domain = irq_domain_create_linear(vpe->fwnode, 16,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) 						   sgi_domain_ops, vpe);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 	if (!vpe->sgi_domain)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) 	sgi_base = __irq_domain_alloc_irqs(vpe->sgi_domain, -1, 16,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) 					       NUMA_NO_NODE, vpe,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) 					       false, NULL);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) 	if (sgi_base <= 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) 	return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) err:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) 	if (vpe->sgi_domain)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) 		irq_domain_remove(vpe->sgi_domain);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 	if (vpe->fwnode)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) 		irq_domain_free_fwnode(vpe->fwnode);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) 	kfree(name);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) 	return -ENOMEM;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) int its_alloc_vcpu_irqs(struct its_vm *vm)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) 	int vpe_base_irq, i;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) 	vm->fwnode = irq_domain_alloc_named_id_fwnode("GICv4-vpe",
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) 						      task_pid_nr(current));
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) 	if (!vm->fwnode)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) 	vm->domain = irq_domain_create_hierarchy(gic_domain, 0, vm->nr_vpes,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 						 vm->fwnode, vpe_domain_ops,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) 						 vm);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) 	if (!vm->domain)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) 	for (i = 0; i < vm->nr_vpes; i++) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) 		vm->vpes[i]->its_vm = vm;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) 		vm->vpes[i]->idai = true;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 	vpe_base_irq = __irq_domain_alloc_irqs(vm->domain, -1, vm->nr_vpes,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) 					       NUMA_NO_NODE, vm,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 					       false, NULL);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) 	if (vpe_base_irq <= 0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) 		goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) 	for (i = 0; i < vm->nr_vpes; i++) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) 		int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 		vm->vpes[i]->irq = vpe_base_irq + i;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) 		ret = its_alloc_vcpu_sgis(vm->vpes[i], i);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) 		if (ret)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) 			goto err;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) 	return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) err:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) 	if (vm->domain)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) 		irq_domain_remove(vm->domain);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) 	if (vm->fwnode)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) 		irq_domain_free_fwnode(vm->fwnode);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) 	return -ENOMEM;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) static void its_free_sgi_irqs(struct its_vm *vm)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 	int i;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) 	if (!has_v4_1())
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) 		return;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) 	for (i = 0; i < vm->nr_vpes; i++) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) 		unsigned int irq = irq_find_mapping(vm->vpes[i]->sgi_domain, 0);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) 		if (WARN_ON(!irq))
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) 			continue;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) 		irq_domain_free_irqs(irq, 16);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) 		irq_domain_remove(vm->vpes[i]->sgi_domain);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) 		irq_domain_free_fwnode(vm->vpes[i]->fwnode);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) void its_free_vcpu_irqs(struct its_vm *vm)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) 	its_free_sgi_irqs(vm);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) 	irq_domain_free_irqs(vm->vpes[0]->irq, vm->nr_vpes);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) 	irq_domain_remove(vm->domain);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) 	irq_domain_free_fwnode(vm->fwnode);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) static int its_send_vpe_cmd(struct its_vpe *vpe, struct its_cmd_info *info)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) 	return irq_set_vcpu_affinity(vpe->irq, info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) int its_make_vpe_non_resident(struct its_vpe *vpe, bool db)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) 	struct irq_desc *desc = irq_to_desc(vpe->irq);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) 	struct its_cmd_info info = { };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) 	int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 	WARN_ON(preemptible());
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) 	info.cmd_type = DESCHEDULE_VPE;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) 	if (has_v4_1()) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) 		/* GICv4.1 can directly deal with doorbells */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) 		info.req_db = db;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) 	} else {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) 		/* Undo the nested disable_irq() calls... */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) 		while (db && irqd_irq_disabled(&desc->irq_data))
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) 			enable_irq(vpe->irq);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) 	ret = its_send_vpe_cmd(vpe, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) 	if (!ret)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) 		vpe->resident = false;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) 	vpe->ready = false;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) 	return ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) int its_make_vpe_resident(struct its_vpe *vpe, bool g0en, bool g1en)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) 	struct its_cmd_info info = { };
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) 	int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) 	WARN_ON(preemptible());
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) 	info.cmd_type = SCHEDULE_VPE;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) 	if (has_v4_1()) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) 		info.g0en = g0en;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) 		info.g1en = g1en;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) 	} else {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) 		/* Disabled the doorbell, as we're about to enter the guest */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) 		disable_irq_nosync(vpe->irq);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) 	ret = its_send_vpe_cmd(vpe, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) 	if (!ret)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 		vpe->resident = true;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) 	return ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) int its_commit_vpe(struct its_vpe *vpe)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) 		.cmd_type = COMMIT_VPE,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) 	int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) 	WARN_ON(preemptible());
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) 	ret = its_send_vpe_cmd(vpe, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273) 	if (!ret)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) 		vpe->ready = true;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) 	return ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) int its_invall_vpe(struct its_vpe *vpe)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) 		.cmd_type = INVALL_VPE,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) 	return its_send_vpe_cmd(vpe, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) int its_map_vlpi(int irq, struct its_vlpi_map *map)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) 		.cmd_type = MAP_VLPI,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) 		{
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) 			.map      = map,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) 		},
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) 	int ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) 	/*
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) 	 * The host will never see that interrupt firing again, so it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) 	 * is vital that we don't do any lazy masking.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) 	 */
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) 	irq_set_status_flags(irq, IRQ_DISABLE_UNLAZY);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) 	ret = irq_set_vcpu_affinity(irq, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) 	if (ret)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) 		irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) 	return ret;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312) int its_get_vlpi(int irq, struct its_vlpi_map *map)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) 		.cmd_type = GET_VLPI,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) 		{
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317) 			.map      = map,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) 		},
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) 	return irq_set_vcpu_affinity(irq, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) int its_unmap_vlpi(int irq)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) 	irq_clear_status_flags(irq, IRQ_DISABLE_UNLAZY);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) 	return irq_set_vcpu_affinity(irq, NULL);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330) int its_prop_update_vlpi(int irq, u8 config, bool inv)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333) 		.cmd_type = inv ? PROP_UPDATE_AND_INV_VLPI : PROP_UPDATE_VLPI,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) 		{
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) 			.config   = config,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336) 		},
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) 	return irq_set_vcpu_affinity(irq, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) int its_prop_update_vsgi(int irq, u8 priority, bool group)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) 	struct its_cmd_info info = {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) 		.cmd_type = PROP_UPDATE_VSGI,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346) 		{
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347) 			.priority	= priority,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) 			.group		= group,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) 		},
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) 	};
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) 	return irq_set_vcpu_affinity(irq, &info);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355) int its_init_v4(struct irq_domain *domain,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) 		const struct irq_domain_ops *vpe_ops,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357) 		const struct irq_domain_ops *sgi_ops)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359) 	if (domain) {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) 		pr_info("ITS: Enabling GICv4 support\n");
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361) 		gic_domain = domain;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) 		vpe_domain_ops = vpe_ops;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) 		sgi_domain_ops = sgi_ops;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) 		return 0;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365) 	}
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) 	pr_err("ITS: No GICv4 VPE domain allocated\n");
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) 	return -ENODEV;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369) }